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TECHNOLOGY AND THE VOTING PROCESS 


THURSDAY, MAY 24, 2001 

House of Representatives, 
Committee on House Administration, 

Washington, DC. 

The committee met, pursuant to call, at 11:00 a.m., in Room 
1310, Longworth House Office Building, Hon. Robert W. Ney 
(chairman of the committee) presiding. 

Present: Representatives Ney, Ehlers, Doolittle, Hoyer, and 
Fattah. 

Staff present: Jeff Janas, Professional Staff; Reynold 
Schweickhardt, Technology Director; Roman Buhler, Counsel; Paul 
Vinovich, Counsel; Chet Kalis, Professional Staff Member; Sara 
Salupo, Staff Assistant; Robert Bean, Minority Staff Director; Matt 
Pincus, Minority Professional Staff; and Keith Abouchar, Minority 
Professional Staff. 

The Chairman. The committee will come to order. I want to 
thank the witnesses for coming to the committee. I would note we 
have several things going on in a conference on the Majority side, 
so the members will be coming in. Also the ranking member has 
a previous commitment, but he will be here; and a couple of the 
other members will also be coming in at different times. We do 
have Mr. Fattah and myself, So we appreciate you coming. 

Our witnesses are Christopher Baum, Vice President/Research 
Director of the Gartner Group, Stamford, Connecticut; Thomas Pal- 
frey, Professor of Economics and Political Science, Co-Director MIT 
Voting Technology Project, California Institute of Technology, Pasa- 
dena, California; David Woods, Professor, Institute for Ergonomics, 
Associate Director of the Midwest Center for Inquiry on Patient 
Safety of the Veterans Health Administration, from my alma 
mater, Ohio State University. I attended there when we used to 
beat Michigan regularly, years ago. And Ronald Rivest, Viterbi Pro- 
fessor of Computer Science, Laboratory for Computer Science, Mas- 
sachusetts Institute of Technology, Cambridge, Massachusetts. 

I want to welcome all the witnesses. As you are probably aware, 
we have had different panels on election reform, and I think they 
have been interesting panels as this has progressed. This is the 
fourth hearing in a series of hearings. Your panel brings a dif- 
ferent, I think, perspective to the hearing process. The first panel 
we had were secretaries of state, State legislators, county commis- 
sioners, disabled community, and American Legion. 

Our second hearing had the local election officials, who were on 
the front line of this, and then the third hearing included the ven- 
dors. And now we have the technology exposition. And now I think 
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from the academic side and the technological side, you will bring 
a different perspective for us. 

Mr. Fattah, do you have a statement? 

Mr. Fattah. Thank you, Mr. Chairman. Let me thank you for 
your continuing role in this to tackle this issue that is critically im- 
portant to the lifeblood of our democratic process. I have an open- 
ing statement that I would like to submit into the record on behalf 
of the ranking member, the gentleman from Maryland, and I would 
like to do that if there is no objection. 

The Chairman. Without objection. 

[The statement of Mr. Hoyer follows:] 
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Opening statement 
Electoral reform hearing: 
Voting technology 
May 23, 2001 


Mr. Chairman, members of the committee. 

This is the fourth healing that the House Administration Committee has held on electoral reform 
in as many weeks. 

Today’s hearing is also the second in the Committee's series on technology and the voting 
process. 

Last week we kicked off the technology series by hosting a 2-day exposition on the latest in 
voting technology, during which 13 vendors demonstrated their wares. 

We heard from the manufacturers who actually build the sophisticated, durable equipment that 
Americans use to exercise their right of franchise. 

1 found last week’s focus on technology to be enormously instructive, for three reasons: 

First, I was able to try a wide range of machines that the voting equipment industry is developing 
now to ensure that our democracy, and democracies all over the world, are conducted accurately 
and honestly: Optiscan, DRE, even voting by telephone. 

Second, I learned about sophisticated software and hardware devices to ensure that voting is 
accessible to all Americans, that votes are counted accurately and completely, and that voters 
have a chance to correct mis-marked ballots before they are cast. 

Third, 1 was able to learn a great deal about- the voting machine industry. Specifically, whether it 
has the capacity to manufacture top-quality, FEC certified equipment in short order - in time for 
the 2002 elections. 

But of course, like any profit-driven industry, the voting machine industry is not a disinterested 
party when it comes to talking about itself and its products. 
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That’s why I am looking forward to today’s hearing. I expect our panel of distinguished 
academics and computer experts to describe in objective terms the quality of voting technology- 
choices today and how they compare with one another. 

I also hope our panelists will comment on how easy and accessible today’s voting technologies 
and ballot designs are for all voters and poll workers to use, based on rigorous research they 
conducted following last November’s election. 

1 will also welcome their thoughts on how research and development of new voting technology 
can best be facilitated by Congress, and how tire system for testing and certification of new 
machines can be improved. 

And last but not least, I will welcome their expert views on just how secure today’s new voting 
technologies are and to what extent they guarantee voter privacy. 

Today’s hearing continues a learning process we began on April 25. It is additional proof of just 
how determined House Administration is to leam what state secretaries, county officials, state 
legislators, local elections administrators, and technical experts are doing right now in the area of 
electoral reform and how Congress can help. 

Chairman Ney and I are already using this knowledge to craft a bipartisan electoral reform 
measure that recognizes the legitimate role Congress can play modernizing our democracy’s 
infrastructure without infringing on the rights of state and local communities. 

/// 
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Mr. Fattah. And I would like to thank our witnesses today and 
I look forward to their testimony. I have a particular interest in the 
Caltech-MIT project, but I am sure that each of our witnesses will 
have testimony that will help us grapple with this issue. Thank 
you, Mr. Chairman. 

The Chairman. And thank you, Mr. Fattah. 

And we will begin with the first witness, Christopher Baum. 

STATEMENT OF CHRISTOPHER BAUM, VICE PRESIDENT/RE- 
SEARCH DIRECTOR, GARTNER GROUP, STAMFORD, CON- 
NECTICUT 

Mr. Baum. Mr. Chairman, distinguished members of the com- 
mittee, Gartner is honored today to give testimony on the trends 
in voting technology. The events of last November point clearly to 
a crisis in confidence in the American electoral system. However, 
there is no clear consensus on how to fix the problem. Some believe 
that electronic voting is the only way to go. After all, it is the same 
technology that worked so well in the banking industry. However, 
there are other opinions. Last week I was present at an election 
site in Boiling Point, Pennsylvania, where voters were invited to 
try a new computerized system in a “shadow election.” after they 
had cast their official ballot on the regular system. I saw one gen- 
tleman who declined to try the new system. I asked him why. He 
said, if you aren’t smart enough to vote on paper, you shouldn’t be 
allowed to vote. 

However, Gartner believes that computer-based voting tech- 
nologies, with appropriate supporting processes and infrastructure, 
can significantly reduce election problems and restore the public 
trust. The public has already demonstrated its faith in electronic 
systems using ATMs to make deposit and filing taxes electroni- 
cally. Computerized voting appliances can significantly improve 
both the balloting and tallying processes, increasing accuracy both 
in the votes cast and the vote count. 

Gartner recently completed a survey on the intentions of the 
United States’ largest voting districts in the aftermath of the 2000 
Presidential elections. The findings of the survey have been sub- 
mitted to this committee; but, in summary, we would like those 
present to consider the following: 

Forty percent of the districts surveyed have already either imple- 
mented new technology or have begun major new voting technology 
projects. 

Those favoring Federal guidance on purchasing and imple- 
menting new technologies are evenly matched with those opposed 
to such guidance, about 45 percent on each side, with 10 percent 
abstaining. 

One quarter of those opposing Federal guidance specifically listed 
implementation problems with the Americans With Disabilities 
Act, ADA, and the National Voter Registration Act, “Motor Voter 
Act,” as examples of why Federal intervention is not warranted. 

However, 50 percent of those in favor of Federal guidance cited 
the need for uniformity in Federal elections. The most cited exam- 
ple was guidance in establishing proper unit trails that can support 
recounts, but do not impinge on privacy. 
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While desires for overall Federal involvement is mixed, 65 per- 
cent favored a specific Federal role in the valuation of new voting 
technology. 

And, not surprisingly, 75 percent reported that their jurisdictions 
would require increased fundings to acquire new technology. 

As a result of this survey, Gartner believes that the local voting 
jurisdictions are willing to purchase and implement new voting 
technologies in time for the 2004 Presidential election, if the Fed- 
eral and State funds are made available by mid-2002. Gartner pre- 
dicts that if Federal and State funds are not available by mid-2002, 
no more than 40 percent of the voting precincts currently using an- 
tiquated systems will successfully implement new voting tech- 
nologies in time for the 2004 elections. 

By comparing the survey results with buying trends, voting sys- 
tem life cycles, voting appliance costs, and projected funding, 
Gartner has created a projection of voting technology for the next 
3 Presidential elections. These projections have been submitted to 
the committee. 

In the short term, optical scan technology will experience a quick 
growth, as it is the easiest alternative to implement, and offers a 
single solution for both polling place and absentee voting. Eventu- 
ally, however, direct record electronic, or DRE, and the closely-re- 
lated i-voting technology also win out. 

There are a number of stumbling blocks in addition to funding. 
First and foremost, is the vast number of different procurement 
procedures used throughout the country. These processes are at 
best cumbersome and lengthy. This is not just another techno- 
logical purchase. There are significant political pressures at work 
as well. For example, Philadelphia recently signed its contract for 
new voting technology in April, capping off a process that began in 
1995. Also daunting, is the fact that no single standard exists to 
certify voting technologies. There is an opportunity for the Federal 
Government to provide guidance and promote innovation in this 
area. 

While this hearing focuses on voting technology, it is critical to 
realize that there is more to an election than casting a ballot. With- 
out rational improvements in voter registration, authentication, 
Election Day planning and operations, and the ballot tallying proc- 
ess, we run the risk of fixing Election Day but not improving the 
voting process. 

Thank you for your attention. 

[The statement of Mr. Baum follows:] 
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Testimony to the United States House of Representatives, House Administration 
Committee 

By Christopher Baum, Vice-President and Research Area Director, Electronic 
Government, Gartner, Inc, 

Distinguished Members of the Committee, 

Gartner is honored today to give testimony on the trends in of voting technology. 
The events of last November point clearly to a crisis of confidence in the 
American electoral system. However, there is no clear consensus on how to fix 
the problem. Some believe that electronic voting is the only way to go, after all, it 
is the same technology that works so well in the banking industry. However, 
there are other opinions. Last week, I was present at an election site in Boiling 
Point, PA. Voters there were invited to try a new a new, computerized system in 
a “shadow election” after they had cast their official ballot on the regular system. I 
saw one gentleman who declined to try the new system. I asked him why. He 
said, “If you aren't smart enough to vote on paper, you should not be allowed to 
vote." 

Gartner believes that the computer-based voting technologies, with appropriate 
supporting processes and infrastructure, can significantly reduce election 
problems and restore the public trust. The public has already demonstrated its 
faith in electronic systems, using ATMs to make deposits and filing taxes 
electronically. Computerized voting appliances can significantly improve both the 
balloting and tallying processes, increasing accuracy in both the voter's cast and 
the vote count. 

Gartner recently completed a survey on the intentions of the United States’ 
largest voting districts in the aftermath of the 2000 presidential election. The 
findings of the survey have been submitted to this committee, but in summary, 
we would like those present to consider the following: 

• 40% of the districts surveyed have already either implemented new 
technology or have begun major new voting technology projects. 

• Those favoring federal guidance on purchasing and implementing new 
technology are evenly matched with those opposed to such guidance, about 
45% on each side with 10% abstaining. 

• One quarter of those in opposition specifically listed implementation problems 
with the Americans with Disabilities Act (ADA) and the National Voter 
Registration Act (“Motor Voter”) as examples of why federal intervention is 
unwarranted. 

• 50% of those in favor of federal guidance cited the need for uniformity in 
federal elections. The most cited example was guidance for establishing 
proper audit trails that can support recounts but not impinge on privacy. 
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• While desires for overall federal involvement is mixed, 65% favored a specific 
federal role in evaluation of new voting technology. 

• Almost 75% reported that their jurisdiction would require increased funding to 
acquire new technology. 

As a result of this survey, Gartner believes that local voting jurisdictions are 
willing to purchase and implement new voting technologies in time for the 2004 
U.S. presidential election if federal and state funds are made available by mid- 
2002. Gartner predicts that if federal and state funds for voting technology are 
not available by mid-2002, no more than 40% of the voting precincts currently 
using antiquated systems will have successfully implemented new voting 
technology in time for the 2004 elections. (0.7 probability). 

By comparing the survey results with buying trends, voting system life cycles, 
voting appliance costs, and projected funding, Gartner created the following 
projection of voting technology for the next 3 presidential elections: 



2000 2004 2008 2012 

Source: Gartner, Copyright 2001 


In the short term, Optical Scan technology will experience the quickest growth as 
it is the easiest alternative to implement and offers a single solution for both 
polling place and absentee voting. Eventually, however, Direct Record Electronic 
(DRE) and the closely related i-Voting will win out. 
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There are a number of stumbling blocks in addition to funding. First and foremost 
is the vast number of different procurement procedures used throughout the 
country. These processes are at best cumbersome and lengthy. This is not just 
another technology purchase, there are significant political pressures at work. 
Philadelphia signed its contract for new voting technology in April 2001 , capping 
off a process that began in 1995. Also daunting is the fact that no single standard 
exists to certify voting technologies. This is an opportunity for the federal 
government to provide guidance and promote innovation. 

While this hearing focuses on voting technology, it is critical to realize that there 
is more to an election than casting a ballot. Without rational improvements in 
voter registration, authentication, election day planning and operations, and the 
ballot tallying process, we run the risk of fixing election day but not improving the 
voting process. 

Thank you for your attention. 



Christopher H. Baum 


Christopher H. Baum VP, Research Director Christopher Baum is a vice 
president and lead analyst in Gartner Research, where he specializes in 
e-govemment (state and local) strategies. He has a strong interest in 
government procurement and electronic voting. Prior to joining Gartner, Mr. 
Baum was vice president, research and analysis director for the data 
networking and healthcare information systems teams at Datapro.Mr. Baum 
earned a bachelor of arts degree in mathematics from the University of Texas 
at Ei Paso and a master's degree in business administration from the 
University of Phoenix. 
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The Chairman. Appreciate your testimony. 

And next, Mr. Palfrey. 

STATEMENT OF THOMAS R. PALFREY, PROFESSOR OF ECO- 
NOMICS AND POLITICAL SCIENCE, CO-DIRECTOR/MIT VOT- 
ING TECHNOLOGY PROJECT, CALIFORNIA INSTITUTE OF 

TECHNOLOGY, PASADENA, CALIFORNIA 

Mr. Palfrey. Mr. Chairman, and members of the House Admin- 
istration Committee, thank you for letting me speak to you today. 
I am Thomas Palfrey, professor of economics and political science 
at Caltech, and codirector of the Caltech/MIT voting technology 
project. My counterpart at MIT is Stephen Ansolobehere who is a 
political science professor there. 

A week after the 2000 Presidential election, David Baltimore, the 
President of Caltech, called up Charles Vest, the President of MIT, 
with an idea. The idea was that our two institutions should collabo- 
rate to develop improved voting technologies. The problems ob- 
served counting the vote in Florida and elsewhere originated with 
technology. 

Presidents Vest and Baltimore assembled a team of computer sci- 
entists, mechanical engineers, process engineers, systems special- 
ists, and social scientists. The Carnegie Corporation and our two 
institutes have funded our endeavors. 

We are in the initial phase of our project, which is a study phase 
and will culminate in a report this summer, which will include a 
number of public policy recommendations. You will hear some of 
those recommendations today. 

First, what is voting technology? Voting technology is usually 
and incorrectly equated with voting machines. In fact, voting tech- 
nology encompasses a wide range of issues related to conducting an 
election, and the Caltech/MIT voting project has been addressing 
all of these issues collectively. Voter registration systems, ballot 
and voter-interface design, security, absentee ballots, voter edu- 
cation, polling place practices, and recounting procedures are some 
of the important components of the overall technology of voting sys- 
tems. 

First, what are some of the problems that we have identified? 
First of all, the high-risk rates of uncounted, undermarked, and 
spoiled ballots, what our preliminary study called residual votes. 
The average incidence of such votes is on the order of 2 percent and 
has been there for about half a century, as far as we can go back 
and measure. Counties using punch cards average the highest re- 
sidual vote rate of about 3 percent, but even if you exclude these, 
we are still over IV 2 percent in residual vote rate. That converts 
to IV 2 million votes or more in the most recent Presidential elec- 
tion. 

Our project identifies what we think is a realistic goal of shooting 
at about a half a percent, because we must understand that some 
of these uncounted ballots are intentionally uncounted in the sense 
that the voters did not actually cast a vote. 

What are some of the other problems? There is a poor monitoring 
in the system. There is very little systemwide data. Something we 
noticed when we began our study is when we began to look for 
numbers, they aren’t all there. So, we basically contacted thou- 
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sands of counties directly to try to get information about under- 
votes and overvotes, absentee ballots, how much they spend on 
election polling place practices, and so forth, and it is not collected 
in the systematic way. 

This lack of monitoring makes it difficult to provide feedback for 
industry and for voting administrators to improve the system. 

There is flawed and haphazard recount procedures. Some States 
have nothing in place. Other States have rules that are so vague 
that it is very difficult to implement in a systematic and uniform 
way. We discovered that with Florida. 

Lack of uniform standards for usability and accessibility. Many 
of the systems that exist now do not conform to typical standards 
that we have acquired under ADA. The user interfaces on some of 
the newer machines — some of the older machines — make it very 
difficult for the voter to figure out how to correctly cast the vote. 
Examples like the butterfly ballot are the most obvious ones, but 
one can find other examples in large numbers. 

Finally, there are shortcomings in the voter registration system. 
The system of cleaning or not cleaning, and purging or not purging 
voter registration rolls, has created problems at the polling places. 

What sort of recommendations do we have in mind? Well, the 
first and most obvious is probably to phase out at some pace — we 
are not sure what the right pace is — what we call dominated tech- 
nologies. These are technologies that are underperforming and do 
not have significant enough cost advantages to justify them. 

The second complementary recommendation is to phase in 
undominated technologies. The current leading technologies right 
now appear to be precinct-counted optical scanning equipment and 
two forms of electronic voting. The first form being what I would 
call integrated touch screen machines. That is, a big box, so they 
are basically modern lever machines, except they are electronic and 
have a touch screen interface, and modular systems which have a 
touch screen, and then it is connected to more off-the-shelf type of 
equipment. So the architecture is actually different. 

All three of these have advantages, however some of them have 
disadvantages. The electronic technologies are really in their in- 
fancy, so innovation is playing a big role in the development of 
these. It is an ongoing process and it needs to be encouraged. 

Third, the transition from old machinery to newer machinery has 
to be done at a measured pace. It is not something that can be 
done overnight. We are talking about a relatively small industry, 
that in a good year, might have $150 million to $200 million in rev- 
enue. We are talking about over 3,000 counties who have strapped 
budgets. We are talking about complicated ways of providing Fed- 
eral funding through grants processes and things like that. 

This is not something that can be done overnight. It has to be 
done in such a way that there is a lot of thought and study that 
goes into it, and it should be done as an ongoing process. 

Finally, along with Federal funding, we think it would be advis- 
able to have set up an independent Federal agency, independent 
from the FEC, that oversees election administration in the country, 
helps administer these grants, administers research grants to re- 
search institutions in universities, and organizes the data, serves 
as an information clearinghouse. 
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Thank you for the opportunity to speak today. 
The Chairman. Thank you. 

[The statement of Mr. Palfrey follows:] 
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Prepared Remarks for 
The House Administration Committee 

Thomas R. Palfrey, Caltech 

May 24, 2001 


Chairman Ney and Members of the House Administration Committee, thank you for inviting me to speak at 
your hearing today. I’m Thomas Palfrey, Professor of Economics and Political Science at Caltech, and co- 
director of the Caltech/MIT voting technology project. My counterpart at MIT is Stephen Ansolobehere, 
Professor of Political Science. Between the two universities we have a team of 1 1 faculty and several 
students, and our central goal is to improve the performance of voting technology. The engineers bring 
expertise in electronic security, user interface design, systems, machine design, and performance standards. 
The social scientists bring expertise in voter behavior, election history, operations management, industrial 
organizational, and public finance. 

Let me start by giving you a little background on the Caltech/MIT Voting Technology Project. Then, I will 
go into some of the findings of our project as they relate to machine performance and directions for 
improvement. Finally, I will comment on the 5 bullet items of the joint press release of Congressmen Ney 
and Hoyer, in regards to the legislation that they are co-sponsoring. 


Overview of the Caltech/MIT Voting Technology Project 

A week after the 2000 presidential election, David Baltimore, the president of Caltech, called Charles Vest, 
the president of MIT, with an idea. Our two institutions should collaborate to develop improved voting 
technologies. The problems observed counting the vote in Florida and elsewhere originated with 
technology. 

Presidents Vest and Baltimore assembled a team of computer scientists, mechanical engineers, and social 
scientists. The Carnegie Corporation and our two institutes have funded our endeavors. 

We are in the initial phase of our project, which I consider the learning phase. Over the last five months 
we have met with many voting machine manufacturers and election administrators to ascertain what the 
problems are and to explore ways that we can contribute to solutions. We have also conducted studies of 
voting machine performance and design, the public finances of election administration, and voter 
registration practices. A report of our work in this phase will be available in July. It will include our 
assessment of existing voting processes in the United States. The report will also offer some specific 
recommendations for the industry, governments, and universities to pursue. 

The second phase of our project focuses on design, and more detailed recommendations for improving the 
system. We’ve identified a number of user interface and security features of existing equipment that can be 
improved upon. We have identified specific practices in voter registration and polling place administration 
that can be improved at minimal cost or with cost savings with the use of computer technology. We have 
also identified the need for a process that would involve industry, government, and universities in continual 
innovation of voting equipment and software. 


What is Voting Technology? 

Voting technology is usually, and incorrectly, equated with “voting machines.” In fact, voting technology 
encompasses a wide array of issues related to conducting an election, and the Caltech/MIT project looks at 
all these issues collectively. Furthermore, there is a strong interaction between technology issues and 
people/process issues that requires simultaneous consideration of both dimensions. Voter registration 
systems, ballot and voter-interface design, security, absentee ballots, voter education, polling place 
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practices, and recounting procedures are ail important components of the overall technology of voting 
systems. However, for this testimony I will focus most of my remarks on those issues most closely related 
to voting machines. 

Performanee criteria 

A voting machine is an instrument, or collection of instruments, for generating, recording, and counting 
votes. In order to guarantee that outcomes of elections reflect the will of the people, and ultimately to 
ensure voter confidence in the democratic process, the voting process has to be designed to work 
efficiently, accurately, and fairly under difficult circumstances. Many criteria must be satisfied. What are 
these criteria that voting machines should be expected to meet? In no particular order, the criteria that we 
have considered in our study are: 

• voter-friendliness; it should be easy to learn to use; 

• poll worker-friendliness; it should be easily deployed, maintained, and stored 

• Secrecy: to avoid vote-buying and coercion 

• Accessibility: make it easy to vote for everyone 

• Standard and reliable ballot design; avoid mistakes like a butterfly ballot 

• Reliabiiity/maintenance; machine cannot fail during performance, or backup must be available 

• Security, opportunities for fraud and tampering should be minimized and detectable 

• Voter feedback; voter should have an opportunity to clearly review ballot before casting it 

• High bandwidth : to avoid lines and reduce cost 

• Error blocking; to avoid overvotes and undervotes 

• Counting and recounting speed; to ensure a timely decision on the outcome 

• Counting accuracy 

• Affordability 

• Audit trail: to permit reliable recounts 

• Indelibility of the ballot; to avoid spoilage or modification after the vote has been cast 

• Upgradability; to take advantage of technological innovations 

Why is this the order of priorities? In making their purchases, election administrators respond to the 
current reality and the priorities and constraints that are set for them by others. Cost is always a problem. 
Voters and the media are impatient for results. Bandwidth and logistical deployment are problematic in all 
major elections. As such, they are high visibility problems and hence high priority items. Contrast this 
with accuracy, security, indelibility, recount process, ballot design and voter feedback. All of these issues 
share the feature that failures are detected rarely — typically only in elections that are extremely close. 
“Virtual ties”, like we saw last year in the presidential race, are not common in mass elections with many 
voters. Largely for this reason, problems of this sort are only occasionally brought to the greater public’s 
attention. For example, we have had a residual vote rate of around 2% for at least half a century. How 
many voters knew this before January 2001? How many election workers knew that? How many voters 
knew anything at all about overvotes and undervotes. When we started our study in December 2000, there 
was almost no published research that systematically compared the accuracy of different voting technology. 
Upgradeability is rarely raised as an issue because significant upgrades are themselves rare events, at least 
under the current system. This is not to say election administrators don’t care about accuracy or security or 
upgradeability, etc. They do indeed! But for practical reasons it often has to take a back seat to the simpler 
objectives like low cost and high speed. 

What are some of the Problems? 


High rates of uncounted, unmarked, and spoiled ballots - what our preliminary study called 
residual votes . The average incidence of such votes is about 2 out of 100 ballots cast over the last four 
presidential elections. These rates vary significantly by state. For example, Massachusetts and Maryland 
have fairly low rates of residual votes - less that 1 percent. New Mexico, South Carolina, Georgia, and 
Illinois have a history of high residual vote rates - above 3 percent of all ballots cast. Some counties have 
had instances of residual vote rates in presidential elections as high as 20 percent or 30 percent of all ballots 
cast. Within states, there is significant variation across counties. For example, in California, Riverside 
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County tends to have much lower residual rates than Los Angeles County. Many factors are at work 
simultaneously, and part of the goal of our study is to identity the leading factors. Some residual votes are 
due to intentional abstention or intentionally spoiled ballots. All the evidence point to this as being very 
rare in presidential elections, with all parties agreeing it is well below 1%. Based on our work, we believe it 
to be less than one-half of one percent. 

Our project has examined what one of our group calls the “epidemiology of voting.” Our intial 
report in March (published in PDF format on our website, http://vote.caltech.edu) examined the extent to 
which the residual vote rate depends on equipment used in the counties. A copy of that preliminary report 
is being distributed to Committee Members as a supplement to my testimony. 

Counties using punch cards average the highest residual vote rate, approximately 3 percent of 
ballots cast. Counties using electronic equipment, primarily full-face DREs, also experience relatively high 
average residual vote rates. Counties using paper, lever machines, and optical scanners average 2 percent or 
less. Within each of these categories, there is significant variation across counties and across time. 

Taken as a whole, these residual vote rates are inexcusably high. Phasing out punch-cards is a 
good first step. But even without puncheards, the national average is over 1.5%. This translates into more 
than 1,5000,000 ballots in the presidential race. Our project identifies a realistic short term accuracy goal 
as a residual vote rate of one-half of one percent, which approximately 10 percent of counties currently 
achieve. Simply using better machines and ballot interfaces is a start, but other factors are involved. There 
are many ways that we can move in the direction of this goal - more poll workers, poll worker training 
programs, voter education, etc. 

Errors in voter registration databases. In response to NVRA a number of states and counties have 
undertaken considerable projects to develop computerized voter registration systems and clean up their 
voter registration rolls. In doing so, these states have estimated the number of duplicate or incorrect 
registrations. Michigan, for example, encountered 1 million duplicate registrations out of approximately 9 
million registered voters. Los Angeles County audited their rolls and estimates that 25 percent of all 
registrations have some sort of problematic or incorrect information. According to responses the Current 
Population Survey conducted by the US Census, in the 2000 election approximately 3 million registered 
voters did not vote because of registration problems. We should set standards for quality of data bases and 
fund efforts to clean the data bases and make these data electronically accessible at polling places. 
Integration of up-to-date and accurate voter registration databases with the actual voting machines is 
another goal that we believe is within reach in the next couple of years. 

Security of electronic voting. Two new technologies - scanners and electronic voting equipment are 
growing very quickly. By 2002 they will cover over half of all voters in the US. Some machines up load 
ballots and transmit votes over the Internet or modems. Standards for securing these transactions are 
required. Also, several counties have experimented with internet based voting systems, such as that of 
VoteHere.Net. Standards should be developed for the security of electronic voting procedures to prevent 
problems and also to foster development in this area. There are also deficiencies in the software 
verification and certification process that need to be addressed. Software verification is a very difficult area 
because bugs (intentional and non-intentional) are hard to detect. 

Usability and Accessibility . There are no unified standards for the ease of use of equipment or 
handicapped accessibility. Many state laws do dictate ballot formats; these affect usability - sometimes for 
the good and sometimes not. A study of state laws as they apply to ballot design is needed as a first step to 
assessing usability requirement. Some uniformity of these standards is desirable, and adherence to such 
standards should be required as a condition for federal funding. 

Auditability. An audit of the vote is needed in cases of challenged elections. The ability to conduct 
such an audit varies across technologies. Audit trails can take several different forms, from dual media 
recording (e.g. electronic with a paper trail), to indelible physical media (CD’s), to delible physical media 
(paper and puncheards), to virtually no audit trail at all. Lever machines and some of the older vintage 
electronics have the lowest levels of auditability— virtually no audit trail at all. Lever machines record a 
tally of the votes for each candidate on the back of the machine. If the device that moves the counter 
forward is broken, then all votes for that candidate are lost; they cannot be reconstructed through an audit. 

If the counter is manipulated or readjusted after the fact (like an odometer), there is no reliable way to 
detect such mischief. Some newer electronic machines make a separate recording of each voting session, 
and CD technology or other indelible media could conceivably be used via peripheral devices. This is an 
improvement over older DREs and lever machines and points to one of the major advantages of electronic 
computer voting technology: upgradeability. However, software problems introduce another possible 
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source of audit failure in these systems. Paper ballots, scanned ballots, and punch ballots have relatively 
high levelsof auditability, but lack a second independent record and also lack indelibility. 

Dominated Technology 

The question of what is the best current technology is not easy to answer, because several current 
technologies either perform very well now, or are new technologies that have the promise to perform well 
in the future. An easier question is: Which technologies currently in use are dominated by other 
technologies? We can identify four technologies that are candidates for this category. We would be 
reluctant to recommend adoption of any of these. 

• Punchcard systems have many known flaws and no advantages whatsoever other than cost and speed, 

• Lever machines and some full-face DREs lack many key ingredients that we believe should be in a 
voting system. Specific weaknesses include lack of an audit trails, maintenance problems, security, 
accessibility, obsolescence, and upgradeability. 

• Centrally counted optical scanned ballots are dominated by precinct counted op-scan. The residual 
rate is much higher, probably due to lack of error blocking 

• Hand-counted paper ballots are impractical except in sparsely populated areas. Besides well known 
security problems (stuffing the ballot or defacing ballots), hand-counted paper ballots lack error 
blocking and indelibility. There is a bandwidth problem with large ballots. It is hard to scale-up to 
large numbers. Ballot storage is expensive and can be problematic. Dominated by precinct counted 
op-scan, which is virtually the same as paper ballots, just not hand counted.. 


Undominated and developing technologies 

Precinct counted optical scanning is among the most accurate. It has other weaknesses, the main one being 
that it may soon be outmoded. If electronic voting technology progresses as expected, optical scanning 
technology could be in the dominated category before long. For this reason, and some other reasons given 
below, we are hesitant to recommend uniform conversion to this system. 

The newer technologies on the block are all varieties of electronic touch-screen voting systems. These are 
highly promising technologies that are in their infancy. Virtually all of the new entrants in the industry are 
in this category. There is no question that this is the next generation, and we need to start preparing now 
for an eventual transition to such systems. They don’t have much of a track record, and the track record that 
exists is mixed. As a result cautious election officials may shy away in favor the older/proven optical scan 
technology. Those who are do shy away should do what they can to keep their options open for the future. 
For example, they should seriously consider relatively short term leasing agreements with foil service 
contracts, so that they will be poised to make the next transition in a timely fashion. 

It is useful to distinguish two significantly different architectures of systems of this category: integrated and 
modular. They are distinguished both in terms of the hardware they use and how they deploy it. Both have 
advantages and disadvantages. Both could benefit from new innovations in the very near future. Modular 
systems have more flexibility and would seem to be more easily upgradeable. 

• Precinct counted optical scan systems . These were the choice in Florida to replace punch-cards. It is a 
relatively safe technology, provides blocking of overvotes, is user-friendly and pollworker-ffiendly and 
allows for recounts. Familiar task. It has some weaknesses. It is not easily upgradeable to incorporate 
newer technology. Paper storage creates problems. Ballot printing is expensive. Ballots can be spoiled 
Possible mechanical failures. 

• Integrated touch screen systems . Ballot design and user interface more complicated. Familiar task. 
Voter and poll worker education required. Expensive. Proprietary operating systems. No multiple use. 
Inflexible. Hardware is the core of the system. “A modem day lever machine.” Audit trail issues not 
folly resolved. 
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• Modular touch screen systems . Same advantages of integrated systems, but without some of the 

disadvantages. Can be put together from standard “off the shelf* equipment. Upgradeable, Multiple use 
could lead to cost savings and efficiency. Software is the core of the system. Virtually no track record. 
Same audit trail issues as integrated systems. Logistical issues with installation. 


Remote and early voting : in temet; absentee; kiosks. 

Internet voting is a hot topic. Absentee voting is a booming business. Both processes deliver convenience 
to those voters who use it. To a small number of voters remote voting it is a virtual necessity, because of 
difficulty or impossibility in getting to the polls. Unfortunately, there are serious security holes with 
absentee voting. Perhaps the most serious is the lack of enforced privacy, which creates easy opportunities 
for vote-buying and coercion (and difficult to detect). Early voting should be considered as a possible 
alternative to “on-demand” absentee voting. Internet voting has all of the problems of absentee voting. In 
addition, there are serious security problems that remain unresolved, as well as accessibility issues related 
to the digital divide. 

Standards and Certification 

We currently expect of voting equipment a minimum level of performance. The existing standards amount 
to minimum criteria for reliability. The equipment must work under a variety of circumstances; it must 
guarantee the voter privacy and anonymity; it must have a very low rate of tabulation errors. 

The standards could be expanded in several ways. First, human testing should be required, since many of 
the weaknesses of existing technology are related to problems of human interface. Second, there should be 
standard requirement related to accessibility. This criterion may not be imposed on all equipment, but 
instead should be a criterion imposed on each polling place. All polling places should contain accessible 
equipment. Not all equipment at each polling place has to be the same. We need to establish standards that 
will lower errors in voter registration databases. We also need standards that ensure the security of 
registration databases and equipment used to access voter registration databases at the polls. 

Specifications mean that the equipment must have an exact set of features, such as a certain sort of 
computer processor or a certain kind of cabling. An extreme form of standards would be uniform voting 
equipment. We do not think uniformity at that level would be easy to implement in the US, or even 
beneficial. However, I see standard specifications as important in the area of security of the vote deposit 
and the tabulation. If these parts of the process, which are largely invisible to the voters, can be 
standardized, then this part of the equipment could be tested and certified separately. The user interface 
and ballot design components could then evolve more quickly and could remain proprietary. 

Related to the concept of specifications is the idea of establishing standard toolkits to streamline some of 
the administrative tasks. For example, for each type of equipment (touch screens, scanners, etc.), ballot 
toolkits could be developed that all companies could use in laying out ballots for jurisdictions. The 
explosion in the number and variety of ballots adds to the lead time in preparing even electronic ballots, 
and can serve as a barrier to entry. Los Angeles County recently held a “bake off’ for electronic voting. 
Vendors were given one month to demonstrate that their machines could handle the 5000 different ballots 
and many different languages in the county. Only one vendor, who had already conducted a pilot test in the 
county, could do so in the time allotted. Common ballot tool kits could reduce these problems. 

Financial issues 

There are significant financial and economic impediments to a rapid wholesale upgrade of our country’s 
voting technology. First, counties bear most of the costs, and they are more constrained fiscally than are 
higher levels of government. Moreover, many of the offices and ballot items are state or federal. Finally, 
voter confidence is a national problem, not a county problem. The federal government should play some 
role, the only question is how big a role. 
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Here is a brief picture of the current situation. This is a remarkably small part of government currently, in 
terms of expenditures. According to our estimate, in the year 2000, all U. S. counties and municipalities 
combined spent on the order of $1 billion on all aspects of election administration. That’s $10 per voter. 
That pays for most aspects of election administration, with state governments picking up the rest of the tab. 
In the counties for which we have detailed budget information, registration and overhead consume the 
lion’s share of this. Equipment acquisition and maintenance accounts for about 10-20 percent and polling 
place operations another 10 to 20 percent. 

The industry is also remarkable for its small size. Total revenues are in the range of $150 million to $200 
million annually. There are significant revenue and cash-flow cycles induced by election cycles. Revenues 
are highly uncertain, with occasional contracts from big counties necessary to keep companies afloat. The 
industry is highly concentrated. The largest vendor has 60% of the market, and the top three have more 
than 80%. Equipment sales are highly decentralized: firms bid on contracts in each of the 3100 counties. 
Some contracts are made at an even lower level. As a result, a major part of the industry is its sales force. 
Expenditures on salesforce limits the resources available for R & D within the industry. Our sense is that 
research on ballot design and user interface design has suffered. Smallness of the industry may also limit 
the profitability of extensive product testing in experiments with human subjects. Finally, except for a few 
new entrants, the business model is hardware-driven using a “sell and manufacture”, although there are 
trends toward lease rather than sale. While speculative, we foresee the industry as evolving in the future in 
the direction of being more software and service-delivery oriented, although hardware leases and/or sales 
may remain a significant component of revenues for a long time. 

Given the current size and sales volume of this industry, it is unclear how quickly or efficiently the industry 
could mobilize to meet the demand created by a sudden injection of money in the form of federal grants. 
Furthermore, if the transition is made too quickly, this would place new ideas, new technologies, and new 
entrants at a disadvantage. The effect would be to dampen innovation, at a time when innovation should be 
encouraged. A preferred approach would involve a more gradual and on-going process for administering 
grants to counties and localities to help them replace deficient technology in a methodical and carefully 
studied way that would create options for future system upgrades or conversions. This will certainly not be 
the last time the country will need to upgrade its voting technology. 

Finally, there are significant financial constraints on the Office of Election Administration in the FEC. 

That office is one of several logical places to perform the sort of information distribution that we see as 
necessary in order to establish best practices and to improve the information that counties have when they 
purchase equipment. An alternative is to create a Election Administration Commission that is independent 
from the FEC. In many ways this independence would make more sense, in order to separate issues of 
campaign finance from those of election administration. In addition to serving as an information 
clearinghouse, such a commission could oversee federal grants to counties for voting equipment, grants to 
conduct research on voting equipment, and head up an office of standards and certification. 

Federal Legislation 


Congressmen Ney and Hoyer outlined five principles that were listed in a recent press release, and which 
were referred to as a guide crafting a final bill. My reactions to these principles are all positive, and are 
given below. We need to upgrade for “the present” (2004) without losing sight of what is further down the 
road. 

• “The elimination of punch-card voting systems” 

Yes, punch-card voting systems need to be phased out. However, we believe this is true of several 
of the other technologies, as well. Machines that have a record of high residual votes, or are obsolescent, 
need to be replaced with newer technologies. At the same time, we advise caution with respect to the pace 
of the transition. If it is done too quickly, it could stifle the recent surge of new ideas and new entrants into 
the industry, and lock us into technologies that will soon be outdated themselves. 

* “The necessity of federal assistance to States and localities that want to replace unreliable or 
outdated equipment” 
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Federal assistance would be of great value. The existing structure of public finance in voting 
administration, particularly with respect to the financing of election equipment expenditures, is in need of a 
fix. It creates systemic problems. Federal assistance, with an appropriate set of incentives and 
requirements, could change this. This assistance should not be “one shot.” It should be spread out over 
time and part of a continuing process. It should be designed to encourage leasing arrangements rather than 
purchase, for efficiency reasons, to encourage innovation, and in order to retain the flexibility to take 
advantage of emerging frontier technologies. The funding should be contingent on adherence to federal 
standards and certification procedures, as described above. 

* “The recognition that the federal government should not mandate solutions. States and localities 
have expertise and practical experience, and the federal government should work with them to 
ensure the integrity of the process” 

Mandating the use of one and only one voting machine everywhere in the country may simplify 
the process and reduce economic costs in the short run, but with significant long run costs of reducing 
innovation and flexibility. Different states and localities need the flexibility to choose from several options, 
and variety is valuable because it encourages competition and innovation. However, the federal government 
should not shy away from its unique role and responsibility for creating and enforcing a uniform set of 
standards and certification procedures, which would include human testing of voting equipment, and 
careful monitoring of performance in the field. A new federal agency, independent of the FEC, should be 
created for these purposes. 

* “The fact that real reform requires more than just a mechanical fix. The federal government also 
can assist states and local jurisdictions by providing funding for voter education, poll worker 
training, the development of a model election code, and research and development grants for 
equipment manufacturers.” 

Yes, it is essential to realize that voting technology reform involves more than just a mechanical 
fix. The new federal agency can play a role in this, and universities can contribute to the research and 
development. Incentives could also be provided that would encourage the development and spread of new 
ideas in the private sector, but these would have to be structured in such a way that they are not just give- 
away programs. Recently, entrants have also been successful in attracting capital in the VC market, as well 
as direct investment from some of the industry giants. 

A key function of the agency should be to serve as an information clearinghouse for data on about 
existing equipment and best practices. This clearinghouse would provide information to states, counties, 
and researchers about the performance and cost of existing equipment. 

I cannot emphasize enough the need for federal funding to develop and maintain standards and for 
certification, including human interface issues such as ballot design and best practices in polling places and 
recount procedures (viz. “the model election code”). 

* “The fact that our election system requires constant maintenance and vigilance” 

Yes, hence the need for a federal agency to oversee the nation’s election infrastructure, to collect 
data about cost and performance, and to provide funding to counties and municipalities as an ongoing 
process rather than as a one-shot infusion of cash. 
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The Chairman. Professor Woods. 

Mr. Woods. Mr. Chairman, Congressman Fattah, and distin- 
guished members of the committee. November 2000 was a vivid 
time in all of our lives, and in the heartland of America as well as 
here within the Beltway. The intense debate following the electoral 
surprise and crisis paralleled debates I have participated in as a 
human factor psychologist and researcher in industries such as nu- 
clear power, aviation, and, most recently, health care. 

The first step in that debate always seems to be assigning blame. 
Some people argued it was a voter error problem. Others com- 
mented on the antiquated technology such as punch cards. Many 
of the young people caught up in the debate were quite intrigued 
by these devices, thinking them only relics of their parents’ ancient 
history. 

My field of human factors studies the interaction of people and 
devices, people and computers, and how these systems sometimes 
fail to have both technological and human components. 

What has our science learned from aviation and health care work 
that we can apply to election technology? First, the difficulties we 
witnessed are not simply voter error. Rather, they are system 
issues in the interaction between people and technology. And I 
want to point out that these interface issues apply just as much to 
the election official interacting with the equipment as well as to the 
voter interacting with the equipment. 

Second, the difficulties we witness cannot be solved simply by re- 
placing apparently antiquated equipment. Replacement systems 
can exhibit poor user/device interface that results in predictable 
risks of error. 

Third, the good news. Many of these issues can be addressed by 
basic bread-and-butter usability, engineering and testing tech- 
niques, techniques that have been developed through our work 
with the Department of Defense and aviation and aerospace indus- 
tries, and today have matured in the computer software industry, 
and are readily available, quick, and economical to apply. 

Fourth, there are unique aspects to the voting context that create 
potentially difficult design decisions and tradeoffs that require 
careful consideration and longer-term investment of our energy and 
innovation skills. 

Let’s go back through these four points a little bit more. It is 
very important to recognize — and we constantly deal with this 
problem after a crisis like the November election — we very easily 
fall back into the blame game. Which is easier? A black eye for 
human intelligence or a black eye for technology? Instead, we have 
to look at the system and the interaction. The failures are in the 
failure design for effective interaction between people and tech- 
nology. 

Second, unfortunately, buying the vendor’s latest model or bring- 
ing in computer interfaces, in and of itself, will not make issues 
and problems revealed by the Florida ballots and electoral con- 
troversies go away. The kinds of problems that were revealed can 
apply to the interaction of people and any kind of technology. 

I also commented that we have a mature research base that is 
available, and a mature engineering base that is available to apply. 
Techniques for usability-testing the prototype designs have ma- 
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tured in the software industry and these can be brought to bear 
very economically. 

Examples of the kinds of principles and techniques we bring to 
bear is the principle of good feedback. If you have to have effective 
device design for interaction, for usability, give people feedback so 
they can see the results of their actions, recognize problems, and 
correct them. The same principle applies to the interfaces with the 
election officials tabulating the vote; provide a visible audit trail. 

With computer technology you can design electronic voting sys- 
tems in many ways. You can even try to copy old paper or lever 
technology over inside the computer system. The change to com- 
puter technology brings potential benefits, but also new pitfalls. 
And this imposes a responsibility on the designers to think through 
how the functions you want to accomplish can break down, and 
how trouble can arise. This requires usability testing. 

There is also the need in the long run for careful consideration 
of the new issues that arise. Balancing security and visibility feed- 
back, providing wide access across a diverse and aging population, 
handling large numbers of issues and ballot choices in a timely 
fashion, supporting recovery for mistakes, and doing it all in a low 
cost, are formidable design challenges. 

I want to point out that adopting new technology may reduce our 
overall average in accuracy or imprecision rate, but create the pos- 
sibility of new forms of failure. 

From past research we also find if vendors’ claim for failure-proof 
designs merit skepticism, as the humorist Douglas Adam quipped, 
“The major difference between a thing that might go wrong and a 
thing that cannot possibly go wrong is when a thing that cannot 
possibly go wrong in fact goes wrong 5 ’ it usually turns out to be im- 
possible to get at or repair. 

So part of usability design is to take into account the possibility 
for error and unanticipated situations. Computerized voting and 
tabulation systems must support our human ability to check and 
detect if new inaccuracies are creeping in. It is easy to rationalize 
away the need for action. Hyper-close elections are rare. My pre- 
cinct didn’t really have a highly publicized problem. It was only the 
usual error rate. 

The Chicago Tribune, in a study, concluded that the error rate 
in Cook County in the last Presidential election had doubled to 6 
percent. I and my colleagues in the human factors profession are 
shocked that we seem so willing to tolerate even that traditional 
3 percent failure rate as a norm. And we ask, where in business 
or transportation or medicine would we tolerate such failure rates? 

Voting is the centerpiece of democracy, and we need to establish 
systems to monitor for the early warning signs that inaccuracies or 
systematic errors are creeping into our voting system. 

In closing, I would like to remind you that technology alone is 
not sufficient. Harnessing the system of people and technology to 
fulfill the ideals of the democratic process calls us all to make a 
commitment to excellence. Thank you. 

The Chairman. Thank you. 

[The statement of Mr. Woods follows:] 
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Good morning Mr. Chairman and members of the Committee. My name is Dave Woods 
and I’m here as a psychologist and professor of ergonomics at Ohio State University 
where I’ve made a career researching human performance in complex environments like 
aviation and medicine. I want to thank you for recognizing that basic principles related to 
human-technology interactions in those environments apply just as well to the 
deceptively mundane environment of the voting booth. This sort of research is often 
referred to as human factors - the study of how people interact with technology and with 
each other. As scientists, we are adept at researching, analyzing and mitigating sources 
of human error. And just as we have drastically reduced error rates in aviation, medicine, 
nuclear power plants, and maritime shipping, I believe we can contribute substantively to 
reducing error and improving the accuracy of voting behavior. 

Partisan politics aside, the reported voting problems in Florida and elsewhere in 
the nation really deserve to be examined in a manner similar to accident investigations. 
For example, in Palm Beach it appears that a well-intentioned administrator altered the 
ballot design to accommodate the district’s aging voter population. But the publication of 
the ballot design in local newspapers was an inadequate simulation of how voters would 
react to the ballot in the voting booth. Therefore, a laudable attempt to make the ballot 
easier for the elderly to read backfired with overwhelming consequences. Some people 
argued the problem was “dumb voters” and it was common to hear the refrain: ‘They 
should have been able to follow the arrows’ or T wouldn’t have done that.’ 

However, previous studies in human performance would likely have anticipated 
such an outcome. The symmetry in the design of the butterfly ballot was recognized as 
problematic, and abandoned, decades ago in the evaluation of nuclear power plant control 
panels. Further, a small-scale high-fidelity simulation (called usability testing) would 
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have detected problems with the "user-friendliness" of the ballot design at a fraction of 
the cost of newspaper publication. Lastly, attempts to take corrective action on Election 
Day (when problems with the ballot were finally detected) revealed a second tier of 
human factors problems that further complicated the situation. 

None of this is meant to cast blame on individuals; rather, it is meant to highlight 
the elements of a system that can contribute to human error. The best solutions in the 
other domains that I have mentioned have emerged from a marriage of engineering and 
behavioral science under the rubric of human factors. However, quick technological 
solutions without adequate input from human factors specialists have actually led to a 
worsening of the very problems they were assigned to address. For example, attempts to 
modernize air traffic control display terminals failed to adequately address human factors 
issues and resulted in the needless expenditure of hundreds of millions of dollars for 
retrofitting and enormous implementation delays. In medicine, attempts to reduce 
medication errors with bar-code technology have actually increased the number of errors 
when the systems have been too cumbersome to use. 

We should be mindful of these examples as federal, state and local officials begin 
to review the cottage industry of new voting technologies emerging as a result of the 
2000 election. Further, we can make no meaningful progress if we play the blame game 
pointing fingers after-the-fact only at the user or only at the design. Instead, we have to 
view the polling station as a system of people interacting with a device to accomplish 
their goal, in this case, registering their opinion about who should govern during the next 
term of office. 


3 



27 


So the first point, from a human factors perspective, is to recognize that these 
difficulties point to basic principles used in the design of other devices to enhance their 
usability and accuracy that can be easily applied to voting technology as well. Termed 
“usability standards” they are already applied in the design of the software controlling 
computer games that our children play. Shouldn’t we apply the same kinds of minimal 
standards to the development of the voting systems and technologies that support our 
election process? 

A second point to remember is that fundamental inaccuracies in registering and 
tabulating votes have been present in our election system all along. However, it took the 
events in November, and an unusually close election, to change our threshold for 
acceptance of those acknowledged error rates. After the fact, we heard that there had 
been many smaller scale “dress rehearsals” for the Florida crisis, but the fact that these 
occurred only in local elections, or probably didn’t make a critical difference to the 
outcome in a particular race, diminished their significance and went unappreciated. 

Some want to regard the problems in the 2000 election as a set of unique local 
events that won’t happen again. We hear this rationalization often when Human Factors 
professionals work within health care or aviation settings. It is more comfortable to focus 
on how “we” are different (our state, our technology, our voters). However, the success 
of high reliability activities like air travel reveals the need to set up mechanisms to learn 
before a crisis occurs. Aviation, and more recently medicine, have done this by 
establishing mechanisms to monitor the early warning signs of systems error analogous to 
those that are creeping into our voting system. 

We can leam from the Florida crisis, and we can act on that new information 
because there is a mature research base on user-centered design. First, researchers on 
human-device and human-computer interaction have studied situations similar to the 
Florida ballot. We have worked out principles to prevent these kinds of problems in the 
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layout and design of devices, for example, in training military personnel as technicians 
and troubleshooters. Second, techniques for usability testing of prototype designs have 
matured in the software industry and can be applied to fit the requirements of 
organizations under budget pressure and in need of making quick decisions about where 
to invest limited resources to make the biggest impact. So a wealth of applied research is 
readily available to help federal, state, and local election officials make design and 
purchasing decisions that will avoid these kinds of election crises in the future. 

But there is a third message I want to leave you with today. I don’t want you to 
think about electoral reform as simply a case of replacing antiquated technology with 
modem technology. Unfortunately, simply buying the latest system or simply bringing in 
computer interfaces will not make all of the problems revealed by the Florida crisis go 
away. The kinds of problems we saw in Florida can apply to any kind of human-device 
interaction, whether that device is a mechanical device or a computerized device. 

The good news is that results from human factors research point to two key 
elements necessary for any human-device interaction to work effectively. One critical 
element is the importance of providing feedback to the user so that they can see the 
results of their actions. Who does the machine think I voted for? Excellent examples of 
good feedback are evident in electronic commerce, look for example at how customers 
order something on Amazon.com. The customer receives feedback in several different 
forms and at several different times during the transaction to verify the accuracy and cost 
of the order. While copying this interface over into the voting booth is probably not 
appropriate, it illustrates the principles of good interface design — principles we can easily 
apply to the unique context of voting. 
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This same principle extends to the tabulation process as well - is there a visible 
audit trail? But feedback is essential not just for the voter but should apply to the entire 
set of election officials involved in the initial tally and any recount. 

It’s also important to note that people may not understand what they can do if 
they feel they have erred or mis-entered something in the voting booth. Would a voter 
fear that leaving the voting booth could invalidate their entire slate of votes if they had a 
difficulty or wanted to correct a mis-entry. How do voters get help to correct a mis- 
entry? Do poll-workers know how to help a voter recover from an error? 

The last issue is an especially important aspect of the voting system and will 
become increasingly critical as Federal and state officials look to Florida’s leadership in 
election reform. Interestingly, Florida just passed legislation mandating the use of 
optically scanned ballots for the 2002 election because of the problems associated with 
the punch card ballots. However, the average spoilage rate for punch-card ballots was 
actually better (3.93%) than the spoilage rate for optically scanned ballots that were not ' 
corrected for errors at the polling site (5.68%). 

But optically scanned ballots were processed using two different systems. The 
more accurate of the two, referred to as "Precinct-Level" tabulation, was designed to 
allow voters an opportunity to correct any errors before leaving the polling station. The 
spoilage rate for ballots tabulated by this method was the lowest of the 5 voting 
technologies used in Florida (0.83%). While clearly an improvement, that spoilage rate 
meant that 17,172 votes were still tossed out and that leads one to wonder what was 
responsible for these uncorrected errors. 
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So as Florida, and other states that might adopt this technology, prepare for 2002 
election officials will need to make a concerted effort to understand why thousands of 
votes were shot down by what probably appeared to be a bullet-proof technology. More 
generally, those responsible for making and implementing election reform policies should 
understand that no matter how sophisticated the technology, voting requires an 
interaction of human with machine, and a poor fit will inevitably result in errors. 

We can apply principles of user-centered design to remedy this situation and 
create something analogous to a buyer’s guide for the states. However, it will require 
some investment over the short time frame before the next election cycle to pull together 
the information most relevant to voting and tabulation processes. We can provide 
recommendations for minimum usability tests that voting systems should pass just as 
buyers guides often review a set of “essential” features for other electronic devices used 
in the workplace or home. 

It is important to use the Florida ballot controversies as a wedge to go behind the 
publicity and identify all of the different places and paths where potential problems and 
inaccuracies can arise. Then we can use that information to learn, modify and 
continuously update our election system(s) to prevent crises like those we experienced in 
November from happening in the future. 

Finally, I would like to close my remarks with one other specific issue highlighted 
in the 2000 election. As we upgrade our technology and change our voting systems, we 
most certainly will change the nature of a recount. I would submit to you that, as we 
change technology and adopt different technologies across different states and localities, 
we will have to accept a variety of recount strategies and set the boundaries for those 
recounts well in advance. 
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In closing, I’ll just remind the Committee that "User-friendliness" happens by 
design, not by accident. Thus when voting-system options are being debated, it will be 
critical to include human-performance as well as technology expertise in the 
deliberations. Although readily available, it is frequently ignored. Thank you for your 
attention. 
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The Chairman. Mr. Rivest. 

STATEMENT OF RONALD L. RIVEST, VITERBI PROFESSOR OF 

COMPUTER SCIENCE, LABORATORY FOR COMPUTER 

SCIENCE, MASSACHUSETTS INSTITUTE OF TECHNOLOGY, 

CAMBRIDGE, MASSACHUSETTS 

Mr. Rivest. Chairman Ney, Mr. Hoyer, and distinguished mem- 
bers of the Committee on House Administration, I thank you for 
the opportunity to testify to your committee on the issue of security 
in voting technology. I have been involved in the mathematical as- 
pects of security for the last 25 years. I lead the cryptography and 
information security within MIT’s laboratory for computer science. 
I am a founder of RSA Data Security, a leading provider of security 
technology. Codes I have developed are used daily to secure mil- 
lions of on-line Internet transactions. 

For the past 5 years, I have investigated the security of elec- 
tronic voting. My students have implemented an electronic voting 
system used for student elections at MIT. I am currently partici- 
pating in the Caltech MIT voting project just described to you by 
Professor Palfrey. Our initial report will be out this summer. The 
opinions expressed here are my own. 

I find voting intriguing. It is not only important for democratic 
society, but it is also technically challenging. 

The challenge arises primarily from the need to remove voters’ 
identities from their cast ballots, in order to prevent vote buying 
and the coercion of voters. This requirement for anonymity makes 
electronic voting different than electronic commerce or electronic 
banking, where well-labeled receipts and well-labeled audit trails 
are standard. This requirement for anonymity can also make fraud 
easier as the addition, deletion, or modification of an anonymous 
ballot is harder to detect. 

In 1869, inspired by the potential benefits of electricity, Thomas 
Alva Edison was granted U.S. Patent 90646 for an electric vote re- 
corder. Congress declined to use it because it reported votes too 
quickly. Today, inspired by the potential benefits of computing and 
Internet technology, inventors and election systems vendors are of- 
fering new technologies. We need to carefully assess what these 
new technologies can offer, to see if they really meet our needs and 
do so securely. 

Given the short time available, I would like to offer some per- 
sonal opinions on the security of existing prospective voting sys- 
tems. I would be happy to expand further on any of these points 
in response to your questions. 

Number one, we are not ready for Internet voting from home. I 
believe that voting equipment should be under the control of elec- 
tion officials. At least a decade of further research and development 
on the security of home computers is required before Internet vot- 
ing from home should be contemplated. 

Number two, however, I believe that we should use the Internet 
to post A lists of registered voters, B lists of actual voters, and C 
lists of actual ballots cast. Not being matched with the voters 
names, of course. 

Number three, as far as getting the biggest bang for the buck as 
far as security goes, I believe we should (A) improve voter registra- 
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tion procedures and the computerization of voter list registration 
lists; and (B) eliminate absentee balloting except for cases of need. 
I am against voting by mail for convenience. I prefer having a na- 
tional voting holiday or allowing voters to vote several weeks early 
at the town hall if need be. Voters who vote absentee are simply 
not guaranteed the same freedom from coercion and bribery that 
ordinary voters have. 

Number four, I believe voting systems should have a physical 
audit trail. That audit trail should be directly created by the voter, 
or at least directly verifiable by the voter when he casts his vote. 
It need not be paper, but it should be immutable and archival. 
Many have proposed electronic systems fail this requirement. Elec- 
tronic voting systems offer improved ease of use and lots of flexi- 
bility, but they do not intrinsically offer improved security. On the 
other hand, a physical audit trail is not a security panacea, al- 
though it is a big help. 

Number five, we must ensure the highest degree of confidence 
that our elections are free of manipulation and fraud. The certifi- 
cation of voting systems should be an important part of this proc- 
ess. 

However, it is difficult to certify complex software-based systems 
involving elaborate user interface and cryptographic functionality. 
Experts in computer security and cryptography need to be involved 
in the certification process. Requiring that all security-critical por- 
tions of the source code be “open source” can greatly help to estab- 
lish confidence in such complex systems. 

We are no more guaranteed protection against election fraud by 
buying flashy electronic equipment, than we are guaranteed protec- 
tion against fire by buying a shiny new fire engine. Security de- 
mands depends on the entire system, not just the components, 
which need sound operational procedures managed by training per- 
sonnel. These operational procedures, which themselves should be 
documented and certified, should primarily ensure that no single 
person or vendor is ever in a position to compromise the integrity 
of our democratic process. 

Finally, I know that we are in the midst of a technological revo- 
lution that provides both an enduring and improving set of oppor- 
tunities, and an increasing set of vulnerabilities. If there is a 
chance to improve things now, then our focus should not be on im- 
mediately spending money for new equipment but, rather, on im- 
proving the higher-order processes of voting system research, evo- 
lution certification, selection, financing, staffing, and oversight as 
well as on improving voter education. 

I thank you for your attention. 

The Chairman. Thank you very much. 

[The statement of Mr. Rivest follows:] 
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Viterbi Professor of Computer Science 
Massachusetts Institute of Technology 


Dear Chairman Ney and members of the Committee on House Administration: 

I thank you for this opportunity to testify to your committee on issues of security in 
voting technology. 

(I apologize for the brevity of these remarks, but I returned home from conferences in 
Europe only Monday night to discover your invitation for my testimony.) 

I have been involved in the mathematical aspects of security for the last twenty-five 
years. Codes I have developed are used daily to secure millions of on-line Internet 
transactions. 

For the past five years I have investigated the security of electronic voting. My students 
have implemented an electronic voting system used for student elections at MIT. I am 
currently participating in the CalTech/MIT Voting Technology Project; our initial report 
will be out this summer. The opinions expressed here are my own. 

I find voting intriguing: it is not only important for our democratic society, but it is also 
technically challenging. 

The challenge arises primarily from the need to remove voter's identities from their cast 
ballots, in order to prevent vote-buying and the coercion of voters. This requirement for 
anonymity makes electronic voting different than electronic commerce, where 
well-labelled receipts and well-labelled audit trails are standard. This requirement for 
anonymity can also make fraud easier, as the addition, deletion, or modification of 
anonymous ballots is harder to detect. 

In 1869, inspired by the potential benefits of electricity, Thomas Alva Edison was 
granted U.S. patent 90,646 for an "Electric Vote-Recorder". Congress declined to use it, 
since it reported votes "too quickly" (!). Today, inspired by the potential benefits of 
computing and Internet technology, inventors and election system vendors are offering 
new voting technologies. We need to carefully assess what these new technologies can 
offer to see if they can really meet our needs. 

Given the short time available, I would like to offer some personal opinions on the 
security of existing and prospective voting systems; I would be happy to expand further 
on any of these points in response to your questions. 
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(1) We are not ready for Internet voting from home. 

— I believe that voting equipment should be under the control of election officials. At 
least a decade of further research and development on the security of home computers is 
required before Internet voting from home should be contemplated. 

(2) I believe that we should use the Internet to post: 

(a) lists of registered voters 

(b) list of actual voters 

(c) list of actual ballots cast (not matched with voter's names, of course) 

(3) As far as getting the biggest "bang for the buck" as far as security goes, I believe 
that we should 

(a) Improve voter registration procedures and the computerization of voter 
registration lists 

(b) Eliminate absentee balloting except for need. 

-- I'm against voting by mail for convenience. I'd prefer having a 
national voting holiday and allowing voters to vote several weeks 
early at their town hall. Voters who vote absentee are not 
guaranteed the same freedom from coercion and bribery that 
ordinary voters have. 

(4) I believe voting systems should have a physical audit trail. That audit 
trail should be directly created by the voter, or at least directly 
verifiable by the voter when he casts his vote. 

— Many proposed electronic voting systems fail this requirement. 

Electronic voting systems offer improved ease-of-use and flexibility. 

They do not intrinsically offer improved security. (On the other 
hand, a physical audit trail is not a security panacea, although it 

is a big help.) 

(5) We must ensure the highest degree of confidence that our elections 
are free of manipulation and fraud. The certification of voting 
systems should be an important part of this process. 

However, it is difficult to certify complex software-based systems 
involving elaborate user interfaces and cryptographic functionality. 

Experts in computer security and cryptography need to be involved in 
the certification process. Requiring that all security-critical 
portions of the source code be "open-source" can help to establish 
confidence in such complex systems. 
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But we are no more guaranteed protection against election fraud 
by buying flashy electronic equipment than we are guaranteed protection 
against fire by buying a shiny new fire engine. We also need sound 
operational procedures managed by trained personnel. These operational 
procedures, which themselves should be documented and certified, should 
primarily ensure that no single person or vendor is ever in a position 
to compromise the integrity of our democratic process. 

Finally, I note that we are in the midst of a technological revolution 
that provides both an enduring and improving set of opportunities and 
an increasing set of vulnerabilities. If there is a chance to improve 
things now, then our focus should not be on immediately spending money 
for new equipment, but rather on improving the higher-order 
processes of voting system evolution, certification, selection, 
financing, staffing, and oversight, as well as on improving voter 
education. 

I thank you for your attention. 



38 


Ronald L. Rivest 

Professor Rivest is the Viterbi Professor of Electrical Engineering and Computer Science 
in MIT 's Department of Electrical Engineering and Computer Science . He is a member of 
MIT's Laboratory for Computer Science , a member of the lab's Theory of Computation 
Group and is a leader of its Cryptography and Information Security Group . He is also a 
founder of RSA Data Security. (RSA was bought by Security Dynamics; the combined 
company has been renamed to RSA Security .') 

Professor Rivest has research interests in cryptography, computer and network security, 
and algorithms. 

Professor Rivest is a Fellow of the Association for Computing Machinery and of the 
American Academy of Arts and Sciences, and is also a member of the National Academy 
of Engineering. Together with Adi Shamir and Len Adleman, he has been awarded the 
2000 IEEE Koji Kobayashi Computers and Communications Award and the Secure 
Computing Lifetime Achievement Award. 

Professor Rivest is an inventor of the RSA public-key cryptosystem. He has extensive 
experience in cryptographic design and cryptanalysis, and has published numerous papers 
in these areas. He has served a Director of the International Association for Cryptologic 
Research, the organizing body for the Eurocrypt and Crypto conferences, and as a 
Director of the Financial Cryptography Association. 

He received a B.A. in Mathematics from Yale University in 1969, and a Ph.D. in 
Computer Science from Stanford University in 1974. 

He has also worked extensively in the areas of computer algorithms, machine learning, 
and VLSI design 



39 


The Chairman. I appreciate the panel’s input. I am going to go 
ahead and yield to Mr. Ehlers first. 

Mr. Ehlers. Thank you, Mr. Chairman. I appreciate that, since 
I have to leave shortly. 

I really appreciate the testimony we have heard, and it brings 
out a number of factors that have always struck me as very impor- 
tant. My background is both having served in local government 
where we deal directly with the election process, and also with 
being a scientist by training and being appalled at some of the 
things that I saw. 

I think the very first step, of course, is good law. That was a very 
major problem in Florida. The law was not well written, well struc- 
tured, it was not clear, and could not be clearly interpreted. I think 
that problem is being dealt with rather quickly by most legislatures 
throughout this land. 

Secondly, we need good technology. I think we have done reason- 
ably well in that in terms of the technology itself, ranging from the 
paper ballot up to the computers. What I find missing in that, how- 
ever, is the next factor we need, and that is good human factors 
engineering. I think that has been a major problem with the newer 
election devices that have been developed and are on the market. 

I also think something we don’t pay enough attention to is good 
security, good privacy, and good integrity. We pay a huge amount 
of attention to that in our computer systems in commerce. Even in 
the House, when we put that system in, that is one thing that I 
insisted on very strongly. And I am very proud, as is Ronald 
Schweickhardt who sits behind me, a staff member involved in 
this, and a number of other members. 

It is interesting to read in the paper about all the hackers break- 
ing into the Pentagon, the Senate, the White House, but you have 
never read about them hacking into the House. I don’t want to say 
that too loudly because then we will become the prime target. 

I am very pleased. We put a lot of effort on security, and I be- 
lieve we have succeeded, but you have to have precisely the same 
security in the voting booth that we installed here. And I don’t see 
that at all in the electronic systems, and even many of the other 
systems are not as secure. 

Just a quick comment on what was said earlier, I think it is im- 
portant for everyone to vote at once, to the greatest extent possible. 
I am prejudiced on that because I run for office and my campaign 
is planned so that all the information is out there by Election Day. 
People vote 3 weeks ahead of time, they miss most of the informa- 
tion that I am providing. But also, absentee ballots lend themselves 
tremendously to abuse and fraud. The one thing we haven’t men- 
tioned here, I think the greatest opportunity for fraud, is in voter 
registration; and we need to pay much more attention to voter 
fraud there and ensuring that voting lists are good, that we purge 
them regularly; that when someone moves, they can’t keep reg- 
istration at their former address and so forth. 

Now, given these facts and some suggestion of research grants, 
what would you recommend the Federal Government spend its 
money on for research grants if we should decide to go in that di- 
rection? Where would you concentrate the efforts in trying to get 
at these various problems I mentioned? 
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Mr. Woods. One of the resources that is available we have al- 
ready, through work with aerospace, the Department of Defense, 
the government has already built up expertise on these human fac- 
tor engineering issues. For example, in the national laboratories, 
they have expert groups who provide advice on these interface de- 
sign issues to many government agencies. We could bring those in 
to, in the short run, provide a great deal of guidance about how to 
evaluate potential new systems and how to implement them and 
train the kinds of election officials in operating these to achieve 
greater levels of success and avoid some of these difficulties that 
could be done very quickly. Some of these organizations could pro- 
vide input to election officials in the form of a guidance document, 
probably on the order of months, and on the order of tens of thou- 
sands of dollars in investment for a short-run benefit. 

Mr. Ehlers. But we have talked in here about educating the vot- 
ers, training the poll workers and so forth. I really would like to 
see a system where that is not a factor. In other words, the human 
factors are so good that you don’t need to educate voters. 

For example, in voting I think you should show the final slate 
and have a Regis Philbin question at the end: Is this your final 
vote? And, if not, you can go back and change it. Poll workers, too, 
they are wonderful people, just the salt of the Earth, really trying 
to serve their country in a very difficult job, which they only do a 
couple of times a year for long hours and hardly any pay, but they 
only do it a couple of times a year. You can’t train them every time. 
And again, I think we need a system that doesn’t require much 
training for poll workers. 

Let me ask a follow up question to that, and the others can com- 
ment on any of these issues. What is the Federal Government’s ap- 
propriate role in this? We don’t run the elections. We have always 
trusted local governments and States to handle that. How would 
we deal with the human factors issue? Are we going to recommend 
certain systems? Are we going to set Federal standards that sys- 
tems have to meet in order to be used to elect Federal candidates? 
How do you see us playing a role in that? 

Mr. Palfrey. I think the Federal Government could certainly 
play a role in setting standards for certification of processes. I don’t 
think the Federal Government wants to insist on everything being 
exactly one way. I think, for example, the current testing proce- 
dures that are done to certify equipment is machine testing. It is 
basically machines testing machines. I think human testing is 
needed for the ballot interfaces and for the various designs that are 
proposed. Currently there is no human testing. I think that is one 
thing that a Federal agency could do is to oversee the testing of 
these machines and the development of appropriate standards. 

Mr. Woods. There are many kinds of guidance documents avail- 
able to organizations that design computer devices for human use. 
For example, several organizations have just put out new guidance 
documents on access to computer and electronic systems for the 
disabled and the visually handicapped. These kinds of documents 
do not tie the hands of designers but, rather, try to be a positive 
resource, to say there are some of the effective techniques and 
ways that you can design your electronic interfaces, as in this case, 
to make access for the disabled more successful. 
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Mr. Ehlers. How do we deal with the security issue? You know, 
if we have the modern equipment of Tamany Hall, any college 
freshman can make some changes to the software on the com- 
puters. How do we really ensure the security of the hardware or 
the software? 

Mr. Rivest. That is a very complex question. I think continued 
monitoring of what is going on, making sure there is separation of 
function, taking apart voting equipment after it has been used to 
see if it has been tampered with, making sure the code is “open 
source,” and looked at by lots of people. It is a multifaceted prob- 
lem. Making sure the poll workers know what is appropriate to do 
and what is not, making sure the equipment doesn’t support modes 
of operation that would allow a poll worker to reset the clock or 
whatever you can do through various kinds of tampering. We are 
dealing with computer systems now that are very much — voting 
systems that are computer systems and have all the complexity 
and security problems of computer systems. We need to keep the 
system simple as possible to minimize the complexity. And security 
often arises from simplicity. So, looking for simplicity in design is 
an important criterion here, too. 

Mr. Ehlers. Do you think it would be reasonable after each elec- 
tion that all the systems are tested, that you run a quick program 
through them to make sure there 

Mr. Rivest. It is reasonable to do some sampling, random sam- 
pling, look at some of them to see if there has been any tampering. 

Mr. Baum. Also, sir, you could apply statistics to the districts 
and find out where there are anomalies and test that equipment 
stronger, and you can harden the equipment that is being used. 
You can also move to internal memories on the computer systems 
that are unalterable so they can’t be changed on Election Day. 
There are a number of technologies that you can apply here to this 
equipment to make them almost military grade in terms of hard- 
ness. 

Mr. Ehlers. What I hear coming through, though, is that you 
see the Federal role as one of doing the research, setting the stand- 
ards, and helping the States and localities to meet the standards. 

Mr. Baum. That is certainly what our survey shows would be ac- 
ceptable by State and local governments. 

Mr. Ehlers. Thank you very much. Thank you, Mr. Chairman. 

The Chairman. Thank you, Mr. Ehlers. Mr. Hoyer. 

Mr. Hoyer. Thank you, Mr. Chairman. Gentlemen, I apologize 
for my lateness. I had another meeting at 11 o’clock, and took that 
as quickly as I could and got here. 

Thank you, Mr. Chairman, for including my statement in the 
record. I won’t repeat it. 

Mr. Baum in the end of your prepared statement, you mention, 
and I quote, “the daunting fact that no single standard exists to 
certify voting technology.” This is an opportunity for the Federal 
Government to provide guidance and promote innovation. 

You have just been discussing that, obviously, in response to Mr. 
Ehlers’ question. Can you describe specifically how the Gartner 
Group found — what it found when it looked at the voting machine 
certification process. 
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Mr. Baum. Yes, sir. What we found was disarray. There is, of 
course, the NASED who will approve voting machines, and anyone 
can adopt NASED-approved machines. But that process, while well 
documented, is still open to a certain amount of interpretation. 

For the States and local governments that do not accept NASED- 
approved machines, there quite often is no set standard. It is con- 
ceivable and I have seen instances where new technologies are 
being introduced, for the companies to call on local sales or local 
election officials. The response is, come back when you have been 
certified by someone, not necessarily who, but just someone. 

That is where the difficulty comes in. There are over 3,000 touch 
points for election systems. One of the things that is very inter- 
esting, is we don’t even have consensus on the number of counties 
there are in the United States currently. We went out and called 
the secretaries of state of each State and got the number of coun- 
ties that they thought they had, and added them up and got 3,066. 
According to the Federal U.S. Census, there were 3,042. So there 
are some fundamental issues here that we really need to address. 
And the lack of standards, you simply cannot survive in a market 
where you have got a $20 million opportunity or $200 million op- 
portunity, and you have to deal with 3,000 separate standards for 
equipment. 

Mr. Hoyer. Could I say this as an aside, not necessarily as a 
question? In the State of Maryland, which is a relatively simply or- 
ganized State in that we have 23 counties and Baltimore City, 
could the discrepancy be that for many purposes Baltimore City is 
considered to be a county, NACo however would presume it to be 
a city? 

Mr. Baum. Right. 

Mr. Hoyer. So that might — I don’t know how many other States 
have that same phenomenon — but that might count for that 20- 
plus discrepancy between the two. 

Clearly, it would be useful from your standpoint, to have an 
agency at some level that looked to assist not in a mandatory way, 
but in an advisory way, to solve this chaos or disarray — I think you 
used the world “disarray,” chaos may be a harsher word — but dis- 
array that you confronted? 

Mr. Baum. Yes, sir. We do not necessarily believe it needs to be 
a separate organization than the FEC, but we do think that there 
needs to be a place where officials can go and where vendors can 
do to say these are the requirements that we recommend, these are 
the procedures that can bring you to those requirements, and these 
are the sources of information. 

Mr. Hoyer. I would say, as an aside, to all of you as you do your 
work, it is my view that the time has come when the election ad- 
ministration responsibilities at the Federal level and the moni- 
toring responsibilities of finances in elections need to be separate, 
not because they are inherently contradictory to one another as the 
OEA currently exists in the FEC, but because there is so much im- 
mediate demand on the financial oversight, that electoral reform at 
the Federal level, as it has at the State and local levels, has taken 
a second or third share. 

Professor Rivest — how do you pronounce that? 
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Mr. Rivest. It is pronounced different ways within my own fam- 
ily- 

Mr. Hoyer. Well, I would like to pronounce it the way you pro- 
nounce it. 

Mr. Rivest. Rivest. 

Mr. Hoyer. You offer some very sobering insights regarding the 
fraud security of new voting technologies, specifically Internet vot- 
ing. Let me say I also agree with the chairman and with yourself 
very strongly that voting is a communitarian process — George Will 
wrote a column on this some months ago — and that coming to- 
gether in and of itself has a value, I think, above and beyond the 
process value. 

You express confidence in using the Internet for improving voter 
registration, but please elaborate on how the Internet can improve 
registration. Obviously, as a number of you have referenced in your 
comments, that is a very key issue with which we need to deal if 
we are going to have voter confidence; that they come to the polls 
and they will be allowed to vote. That, coupled with a very good 
provisional voting process. But would you comment on that? 

Mr. Rivest. I would be very happy to comment on that, Mr. 
Hoyer. I think at the highest level, the voting system will be im- 
proved by a greater degree of transparency, having more informa- 
tion available to more people, more eyes looking at the process. Ap- 
plying that to the voting registration process, if the voting registra- 
tion lists are posted on the Internet in a way that anybody can look 
at, you will see fewer dead people on the rolls, you will see fewer 
people that are still on the rolls that have moved. You know, some- 
body will call up their official and say, did you know that so and 
so has left town? Things like this can happen. 

So just the process of keeping the accuracy of the voter registra- 
tion lists will be improved by having it open and public, as I believe 
it should be. 

I think the process of registering to vote per se, is not something 
we should be attempting to do over the Internet. I think it should 
require an in-person visit. But once you are on the registration 
rolls, having your name listed as you are registered to vote in this 
county, I think that would be a help. 

Mr. Hoyer. Do you have any thought as to how long it would 
take to construct a statewide system in a State like California, 
which is — well, there are no States like California. That was a stu- 
pid thing to say. There are only a few Nations like California. But 
a State — let’s say a smaller State, Florida, has just gone to man- 
dating a system of central registration. I don’t know what cost that 
they attributed to that. Maryland is going to a central registration 
system. I think that is going to be critical, because in an era of very 
mobile individuals and families, when you move from precinct to 
precinct, house to house, and you really don’t — there is nothing 
that tells you this is a precinct line — that we need to have that 
kind of system. 

What is technically — how long would it take to create such a sys- 
tem in a State like Maryland or Florida or Pennsylvania, and what 
would its cost presumably be? Anybody have a guess on that? Have 
you looked at it? 
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Mr. Rivest. Let me just respond. Part of it is different States do 
it in different ways. I think it is Michigan that is combining it with 
the DMV database. If you have a single voter registration database 
and that is its only function, you might be able to do it in 2 or 3 
years. And I don’t know what the cost would be per voter. But if 
you are trying to organize your State’s citizen records along more 
lines than one, so you are doing the DMV and the Social Security 
or whatever, everything else altogether with within the State, I 
think that could be 6 or 7 years and lots more money. 

Mr. Hoyer. Professor Rivest, one of the things, though, if you try 
to combine it, your transparency issue becomes more complicated, 
it seems to me, because there is information that voters clearly do 
not want and should not want transparent to the rest of us. 

Mr. Rivest. Yes. 

Mr. Hoyer. So that from my perspective, you are really going to 
have very limited use of a document that is totally transparent, 
and I agree with you it ought to be totally transparent, because I 
think that will protect against fraud and mistakes. I think my own 
view is that there are far more mistakes than there are actual in- 
tentional fraudulent acts. 

Mr. Baum. Sir, the National Voter Registration Act of 1993 pret- 
ty much requires a strong link with the Department of Motor Vehi- 
cles in each of the States, so that has to be investigated. Even so, 
there are commercial products available already. At its most simple 
level, a statewide voter registration system is simply a database. 

Now, there are security issues that surround it, but they are all 
known and solvable. And when you are taking a look at a State 
with the population of New Mexico, or something like that, it is not 
even a large database, so these are — again, it is more of a political 
consideration than a technical consideration. 

Mr. Hoyer. Well, I am hoping that one of the things we do in 
legislation, that we are considering, is to have dollars available to 
assist States in the creation of a central database. In my view, in 
large part, the States will have to decide the individual problems 
and how they create that; that we can assist them in doing that 
to provide greater voter confidence when they go to the precinct, 
somebody will know in this computer age, when everybody gets on 
the Internet, can get access to gargantuan amounts of information, 
that they can get information that Joe Dokes has in fact registered 
to vote in this State. 

So, Mr. Chairman, I have got other questions but I know that 
time keeps running. Do you have questions? 

The Chairman. I have got questions. 

Mr. Hoyer. Why don’t I yield to you? 

The Chairman. And if you want more time. I want to ask a spe- 
cific question, Mr. Palfrey. Are you aware of any research that has 
studied the voting patterns of persons who are dyslexic or, let’s say, 
illiterate? 

Mr. Palfrey. With respect to the voting technology? 

The Chairman. And how to facilitate and help those individuals. 
Or is there technology that — for example, technology comes forth — 
is it helpful, not helpful? 

Mr. Palfrey. There is some informal research. I don’t think 
there has been very much that has systematically been done to 
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identify specific problems with technologies. Certainly, with respect 
to user interfaces, maybe Professor Woods can say more about this 
as far as dyslexic interfaces, but I think as far as handicapped con- 
siderations, use of audio, use of Braille and also overlays for the 
optical scanning equipment. 

The Chairman. We know, if it is an issue of sight, I saw the 
equipment here. So we know you have the earpiece. If it is an issue 
of height, you can accommodate downwards. But if it is an issue 
of dyslexia or an issue of illiteracy — 

Now, an easy answer is, we take somebody in the voting booth, 
a poll worker. For a lot of people, though, it is such a private item 
about voting. So I just wondered — again, for sight, there are certain 
devices, but what about dyslexia? 

Mr. Palfrey. Well, there are examples 

Mr. Hoyer. Can I make a comment, because I would like to 
hear — everything you mentioned, the hearing — that the audio 
transmission may not only solve the sight problem but may solve 
the inability — the dyslexic problem or another reading problem 
that might occur. That might solve, most of us wouldn’t think of 
it in those terms, but it may do that as well. You might want to 
comment on it. 

The Chairman. And touchtone screen. It was an interesting issue 
that we were discussing today. 

Mr. Palfrey. Right. And that is a possibility in some foreign 
countries, where illiteracy is a more serious problem than it is 
here. In South America, for example, they actually have pictures 
of the candidates, or icons to represent different parties, for exam- 
ple, to assist voters who can’t read. 

The Chairman. Also, Professor Palfrey, in your study over a 20- 
year period, 60 percent of the counties have adopted new tech- 
nologies of one sort or the other. Forty percent of the counties have 
been using the same technology basically, I would assume, for 20 
years. Are there any similarities or common characteristics about 
counties that have been using the same technologies for a long pe- 
riod of time versus the ones that have switched to some new tech- 
nology? Are there any outstanding 

Mr. Palfrey. Yes, there are two factors that seem to be impor- 
tant with that. The larger counties, populationwise, we have tend- 
ed to switch to new technologies, largely to handle problems of 
bandwidth and just dealing with large numbers of voters. So, they 
are concerned about speed and cost. That actually was one of the 
reasons for the transition even earlier to punch card systems, but 
certainly transitions from paper ballots to optical scanning equip- 
ment was done for cost and speed considerations in larger counties. 
There is also a correlation, with sort of average income levels or 
the sort of county revenues controlling for size, that counties that 
don’t have as much to spend, available to spend, have not been 
changing as fast. 

The Chairman. Mr. Baum, I wanted to ask you a question. You 
had stressed, and I think made a couple of good points about, the 
fact of the different standard where somebody turns to an election 
official and they say, “as long as you are certified.” 

Mr. Baum. Right. 
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The Chairman. Let me just throw something out here, because 
you mentioned the word “agency” a couple of times, as other people 
indicated. Let’s assume you are not going to create an agency per 
se, as we would think an agency to be, because the downside of 
that, frankly, in the election process, is that it is not an environ- 
mental issue or a highway where you have a daily ongoing situa- 
tion. So therefore, the thought of agency would tend to scare people 
across the board here, I think on both sides of the aisle, and a ma- 
jority of both, only because a downside could be that there would 
be constant rules made by a rulemaking body that would make 
people eventually shiver about the agency saying whether you have 
an ID at the poll or not, versus, you know, the local governments 
and the Congress. 

So let’s assume we don’t have a full-blown agency. But what 
would you — or could you speculate, on what you would want to see 
if we created some type of body that was an advisory type of body; 
who would be on that body, in your opinion? 

Mr. Baum. Well, first I would like to respectfully suggest that it 
is certainly more than a one-day project or a single-day project. The 
example I like to give most is that of a wedding. When you start 
planning a wedding, the events of the day actually become very 
much smaller in proportion to what it took you to get there. And 
running an election from an election board’s point of view is very 
much like planning a wedding year after year after year. 

The Chairman. I should clarify. I know it is year-round. I am 
just saying, if you had an agency, though, and you set up a full- 
blown agency, if it does have a daily function, I think we would be 
scared of the product after about a year. If you have an agency that 
creates a new staff daily, thinks of new rules to create on local peo- 
ple, I should tell you where I am coming from on that. 

Mr. Baum. So in that setting, then, I think that what you need 
is representative — I think what we are talking about here is really 
a governance board, not a new kind of committee, but people that 
are involved with the process. And that means that you have peo- 
ple from State and local and Federal elections; you have people 
that have other concerns, interest groups, people who come in that 
are proponents for those who have disabilities; making sure that 
people that come in, from the Majority and Minority parties, to en- 
sure that ballots are fair in the way that they are set up, and peo- 
ple with other interests coming in — I was going to use the term 
“disinterested,” but we are all voters, we should all be interested. 
Like my fellow panel members here; people that can come in and 
come together and build a consensus on these, including the ven- 
dors themselves. 

The Chairman. Thank you. 

I had a question of Professor Rivest. For the machines you are 
familiar with — because Mr. Swigert tells me he wouldn’t let you 
bring a laptop in here. That is how good you are. So those voting 
systems which you are familiar with, how long would it take to 
break into them while in a polling place? That could change the re- 
sults of an election, if that could be done? Is it a long time process? 

Mr. Rivest. I think it depends on who is doing it. I think if you 
have a voter who is just walking in to try to, you know, monkey 
with the equipment he is probably unlikely to have much effect. A 
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poll worker, similarly, has little background with the equipment 
and wouldn’t know quite what to do except to operate it. A vendor’s 
technician on-site might have intimate knowledge of how the ma- 
chine works and might have a preprogrammed smart card to 
change a tally or something like that. 

So I think it depends on knowledge of the equipment. It could 
be very quick if someone knowledgeable — changing electronic sys- 
tems, you know, can be done quickly. If there is a card that is 
plugged in, as many machines have, that controls the operation of 
the machine. If the card can be changed you are changing the pro- 
gramming of the machine. If there is no audit trail, if that card 
was removed and replaced, you may have a successful hack. 

The Chairman. Some observers have recommended an open 
source process, so software can be used to control voting systems 
that would be subject to impartial inspection that could lead to in- 
creased quality. What do you think about that idea? 

Mr. Rivest. I am in favor of it, basically. I think that this poll 
process is too important to be left wrapped in the cloaks of the ven- 
dors’ claims of proprietary need. I think that having the software 
available for inspection by whomever will increase our confidence 
that the voting is being tabulated correctly. 

It is not sufficient by itself, of course. You need to assure that 
the software that you have looked at is actually the software that 
is running in the machine and have procedures installed to ensure 
things like that. But having the software out for inspection I think 
will be to everyone’s benefit. 

Mr. Chairman. The last question I have is for Professor Woods. 

We had the voting machines in here, and I found them of inter- 
est because that is the first time in my life I have not used a punch 
card. I went originally to the X’s we used to do when I first voted 
down in Ohio in Belmont County. Then we went to the punch 
cards, because that is what we used. So this is the first time I ever 
actually touched some other type of device. 

The one thing I would ask of the vendors is, don’t help me, let 
me see if I can do that. I don’t claim to have the ability of Mr. 
Ehlers on scientific knowledge at all our computers, but it was in- 
teresting just using the machines. 

Now, I noticed an exhibit was there when, once you wanted to 
make your choice, you had to retouch to cancel out and then you 
touch back. But then there was another machine that you pushed 
the next name, it automatically canceled back. Because of what you 
do, dealing with human interface and ergonomics, is there any ma- 
chine, when you talk about the touch screens or maybe a non-touch 
screen, that is a better type of machine or the machine of the day? 

Mr. Woods. It is difficult for us to recommend a particular tech- 
nology or system. But the example you use is I think a very illus- 
trative example where some of the principles we found apply. One 
is feedback. So either of those systems can confuse people depend- 
ing on the degree to which they get clear feedback with respect to 
their actions. 

So, for example, take the second case you have mentioned before, 
where you simply push the second candidate. Someone could get 
confused about what the impact of that is. Are they going to end 
up the equivalent on the punch card of having two holes punched? 
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If you have clear feedback — for example, when you press the sec- 
ond one, the first one starts to blink in a salient way and then goes 
dark — those kinds of systems are likely to work better. 

The second aspect of my response, this is the sign of usability 
testing, is about that we have procedures for doing tests so we 
don’t rely just on opinion or quick walk-up, a couple users trying 
it out. We have processes to do that that are very economical to 
generate the best kind of information to act on and get the best le- 
verage when you have to make design decisions or purchasing 
choices. So by using that technology of user testing, technology that 
is used every day in the software industry for the products you and 
your family buy on the local scene, those kinds of tests can result 
in many of these kinds of questions and result in more easy to un- 
derstand and use, especially for the diverse population that you 
pointed out is so critical to the voting application. 

The Chairman. One final thought — I will see if Mr. Hoyer has 
some questions — basically indicated here and at least one of the 
testimonies that, you know, it is the human error. You know, we 
know that is part of it. But if it is just a human error factor, should 
we just keep punch cards and spend a lot of money? Well, if it is 
only human error, could we keep those punch cards and spend a 
lot of money to educate people, or should we go on to something 
else? 

Mr. Palfrey. With respect to punch cards, I think we should go 
on to something else. 

The Chairman. Your testimony did indicate, though, that it is 
human error. Well, if it is, do you just work with what you have? 

Mr. Palfrey. One way I like to think about it is in terms of a 
technology being dominated. In other words, if you think about 
spending the effort to educate the voters, if you want to spend the 
same amount of effort educating voters using punch card systems 
or you are educating voters using an optical scanning system, what 
would the results be? And I think the results of one would be bet- 
ter than the other. 

The Chairman. Is there anyone that would like to keep punch 
cards? 

Mr. Baum. I think punch card manufacturers would like to keep 
punch cards. 

The Chairman. We know that. 

Mr. Woods. The success we have achieved in cockpits and other 
high criticality domains has come from dropping this red herring 
of human error. When we want to label a problem as human error, 
the scientific approach says you take that as a symptom of an un- 
derlying problem in the system and the interaction between people 
and technology. And there lies the grounds for improvement so that 
we can achieve what we all desire, in this case in the democratic 
process. 

So, now, with respect to punch cards in particular, it violates a 
very old rule for designing interfaces between devices and people, 
all right? The punch card, you actually make a hole, all right? The 
absence of something, the hole is supposed to indicate the presence, 
the state that we are interested in, the vote. 

If I had designed — when I started our designing nuclear power 
plant control rooms 20 years ago, if I had tried to code the critical 
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state stage of the nuclear reactor using that principle that is done 
in punch cards, I would not be here today. We have learned 
through bitter experience in aviation interfaces and energy systems 
that is not a good way to set up the interface. 

The Chairman. You are making a good point. So I totally grasp 
it, it is not a good way to set it up to read that card, or it is not 
a good way to set up to have the human being, the voter, reacting 
that way of the punch? 

Mr. Woods. Both. 

The Chairman. I understand, on the reading side of it, I believe, 
of the machine reading it — it is probably not the word — but why 
the voter? 

Mr. Woods. We say that in all these issues of did they punch 
through, was the card aligned, the basic rule of thumb is you want 
people to make a positive indication so that you have transparency 
and traceability, as my colleague to my right indicated earlier. 

The Chairman. Do you have any questions? That is interesting. 

Mr. Hoyer. Well I think that — we pretty much have a consensus 
I think in the country at State, local and Federal levels that we 
want to get — rid ourselves of the punch card, save perhaps the 
manufacturers, as Mr. Baum put it. But it is interesting that sys- 
temically there is a reason for that as well. I was glad to hear that. 

Let me ask Dr. Rivest, then I am going to go to some other quick 
questions. You have reservations about the Internet voting, and 
that is shared I think by a vast number of people, including myself. 
However, I would like your comments on whether it is worth test- 
ing in an area that we think does not work as well, and we need 
to make it work well and may be an area where we want to at least 
test this, and that is overseas voting and military voting. Could you 
comment on applications that may be perhaps more appropriate in 
that limited setting? 

Mr. Rivest. I would be happy to. 

I think that the military application is a good one to look at. I 
think that you have the kinds of controls both over the environ- 
ment and the networks that you may not have in other kinds of 
remote Internet voting, and you also have a very clear and impor- 
tant need for our soldiers to be able to vote, and also I am in favor 
of experimentation. 

I think we need to plan to evolve our voting systems over the 
next 20, 30 — you know, as technology improves, we are going to 
keep changing our minds as to what, as Professor Palfrey said, 
what the dominating technology is. But in order to learn what the 
dominating technology is, what the best voting systems are, we 
need to have experience with them. 

So, for all of those reasons, I think that experimenting on a small 
basis at first, maybe expanding if the system seems to work well, 
with remote Internet voting for military purposes, I would be in 
favor of that. But it needs to be carefully controlled and looked at. 
The Internet is a very fragile and vulnerable entity, and it is vul- 
nerable to attack by malicious organizations from outside our coun- 
try. So we need to make sure that there is back-up systems in 
place should our soldiers not be able to use the Internet. 

Mr. Hoyer. Thank you. 
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Professor Palfrey, I was pleased to see in your statement that 
you support the five principles that Chairman Ney and I are essen- 
tially working around; and I take it from your statement that our 
focus should not be exclusively on the 72,000 precincts, or whatever 
number precinct we have, the 72,000 precincts, but on a broader 
array of the technology use in other technologies, used in other pre- 
cincts, is that correct? 

Mr. Palfrey. That is correct, yes. 

Mr. Hoyer. How expensive do you think we ought to be on this 
in terms of assistance to improve an election infrastructure? 

Mr. Palfrey. Well, I think as far as coming up with a single 
number I am not sure that is quite the way that I think about it, 
partly because I view this as sort of an ongoing process. As far as 
approximately, I don’t know if you are looking for approximate 
numbers. 

Mr. Hoyer. No, I was not looking for a number. I apologize if I 
implied that, obviously. 

But the bill that I have introduced, along with a lot of other peo- 
ple, sought first, as Florida has done, to eliminate the punch card 
system. And then, as you know, it provides ongoing resources for 
technology replacement as well as a lot of other things in terms of 
education and technology development. 

What I am wondering is, if we limited our first step to that, 
would you think that is an error, or do you think we ought to make 
sure that this can be applied to other technologies, i.e., lever ma- 
chines initially? Obviously, we are talking about limited resources 
that we will come up with in the first traunch, if you will. That 
is what I am really getting at. 

Mr. Palfrey. I think there are other technologies that have 
known problems besides punch card systems. Without identifying 
specific vendors, there are examples of full face DRE machines that 
have not been successful, that the administrators that use them 
would probably be happy to replace. I think centrally counted pre- 
cinct scan equipment should be replaced by precinct counting. If 
you are looking for a quick fix of that, that is certainly a better 
technology having to do with the voter feedback and also the ability 
for errors to be detected before the vote is tallied. 

Mr. Hoyer. so in other words — and I hadn’t thought about 
that — but clearly everybody believes that in order for a system to 
be as accurate as it can be you need precinct where-a-voter-is-still- 
present technology to let the voter know whether or not they have 
made mistakes. 

Mr. Palfrey. Yes. I think we are at a stage now where there ex- 
ists technologies to do that. We should take advantage of it. 

Mr. Hoyer. Okay. That is helpful. 

Dr. Woods, you addressed three key factors of voting — user 
friendliness, voter feedback and balance the design. What is the 
best feedback tool to let voters verify whether they actually voted 
for the candidates that they want without compromising voter pri- 
vacy? 

Mr. Woods. Well, you have hit the heart of the design problem 
from a human factor’s perspective. 

Normally, as Professor Rivest indicated, we don’t have those si- 
multaneous constraints to deal with. Obviously, the feedback has 
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to be something that can be terminated by the voter so that they 
get a display, they get feedback that says this is what the machine 
thinks you told it. Then, in registering their vote, the voter must 
have a confident feeling that that information has gone into the 
computer and the traceability back to them has gone away; and 
that requires careful design and testing with users, again meeting 
the challenge of the diverse kinds of populations we have in the 
voting. And that is the kind of usability testing that is standard 
in our profession and that we can carry out on a very rapid basis. 
But there is no way to give a global answer without going through 
that indurative testing process. 

Mr. Hoyer. Okay. Obviously, that, however, is going to be one 
of the key issues we need to deal with; and I say that perhaps not 
at our level. We are talking about a lot of things that, frankly, 
States and locals are going to make the decision — final decision on, 
perhaps with advice of counsel and best practices and best stand- 
ards advice, but 

Mr. Woods. This is where an independent technical group can 
come in and demonstrate. Often, we can show you some clearly un- 
desirable ways to try to accomplish those goals, and we can point 
people in these kinds of guidance documents I have referred to to 
several different kinds of techniques that will work and how to do 
quick tests to verify that your particular choice as an election offi- 
cial or a State official will be successful. 

Mr. Hoyer. Thank you. 

Dr. Woods, let me ask you about the DRE machines, because I 
want to know what extent the flaws in DRE machines are attrib- 
utable to poor ballot design versus problems that are inherent in 
the technology itself or perhaps a combination of the two. I think 
Dr. Palfrey just made the observation that DRE machines them- 
selves — some technology applications have not worked as well and 
administrators are not pleased with how well they have worked. 
Could you comment on the issue? 

Mr. Woods. I haven’t looked directly and evaluated those inter- 
faces. Ohio State did run a study 2 years before the recent con- 
troversy evaluating a variety of different voting systems and antici- 
pated many of the punch card difficulties and identified a variety 
of other problems with the visual layout of the ballot design, ways 
that people could miss certain aspects of the ballot choices they 
had. For example people were able to — we had some issues where 
people — shorter people missed certain referenda, didn’t even realize 
that they had those options available to vote on them. That is why 
there was no choice made on those issues facing the voter. So those 
layout issues can apply. 

Mr. Hoyer. Anybody else have any comment on that? 

Mr. Palfrey. I think actually Mr. Ney identified one source of 
some of these problems when he mentioned the machines that he 
looked at and there was all sorts of variety of ways that lights 
popped on and went off and whatever. And if there isn’t good, you 
know, clear instructions on the screen for what is going on, voters 
could easily get the wrong idea. 

Mr. Hoyer. Seems to me you need to take this technology and 
expose it to large focus groups and make a scientific analysis of the 
human error rate, user friendliness of the technology. I think that 
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is going to take a lot of — I imagine you folks are the perfect people 
probably to do that on contract for some group, perhaps the OEA, 
which would be able to give grants to carry out such testing. I 
mean, ultimately, a human being is going to use this, got to put 
a lot of human beings in the room and see what happens. 

Mr. Palfrey. Not only sort of human testing, sort of human lab- 
oratory testing where you put them in a room and have them bang 
around it, and also field testing could be useful as well. 

Mr. Baum. In addition to in-field testing, we are not really talk- 
ing about voter education here. That is an oxymoron. You don’t go 
out and educate voters on the technology. They come in and use it. 

Mr. Ney, you indicated that was your first time seeing that tech- 
nology. For a majority of voters the first time they are going to see 
that technology is when they go in to vote on it. That is incorrect. 
They should have had a familiarity with it before. You put it out 
in the shopping malls, you put it out in the libraries, you put it 
out in the high schools and let them go out and play with it some. 
So by the time they go in and see it, it is not an intimidation of 
the equipment. 

The Chairman. Especially for certain age years now. My chil- 
dren, if I buy a VCR today, they install it, my 12-year-old. It used 
to be my 17 year old when he was 12. If I need something done 
on the computer, they come over, and they help me. I am the prob- 
lem. They aren’t. 

So I know you know the school age children through computers 
through school are not the — that they are not — that they don’t need 
the education. They will have a much easier time. They don’t fear 
things as I do. I am not a user of technology. I try and, you know, 
I do e-mail and things like that. So there is also age groups. 

That is why you have got a great idea, the malls. If you have a 
mobile traveling unit that goes to senior centers, it is you know, 
open to the public. The schools are great, too. But school children 
are going to adapt quicker because they are used to computers. 

Mr. Baum, it depends. I think my kids would be very intimidated 
by a punch card machine 

The Chairman. Yeah, that is right. 

Mr. Baum [continuing]. And have a whole different experience 
with what the marked sensor cards are, too. The optical scan, you 
look at one of those, and you are taking a test. 

Mr. Woods. We refer to this problem as the walk-up interface 
problem, because people won’t have practice. Even if it is you are 
voting in the same area, you only do it once a year. So it is always 
this kind of walk-up interface, and you have to remember again 
how to do it. 

I would point you, without even the latest technology, to the L.A. 
Olympics in 1984. One major telecommunications company in the 
U.S. Human factors group was — volunteered to help design the in- 
formation system — the electronic information system for the ath- 
letes, the participants in those games, how to deal with people with 
different languages and backgrounds and compute awareness. And 
they needed information about the times of their various practices 
and preliminary heats and also how to contact and schedule meet- 
ings with that tremendous opportunity to get to know people from 
around the world. 
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It was a tremendous success because they applied the science of 
human factors to those walk-up interfaces and experienced very 
few problems. People didn’t think about it as a problem or a chal- 
lenge, how to learn how to operate the device. They just walked up 
and found the information or communication to the people they 
wanted to contact. 

The Chairman. I think Mr. Hoyer made a good point about focus 
groups. You have the scientific community that watches the group’s 
interaction, you have your different type of people that are watch- 
ing, but have you the people that participate in focus groups? I 
think that in fact, you know, would help a lot. 

Mr. Woods. Those usability tasks can be run. 

Again, I would point out there are plenty of interfaces and poten- 
tial for problems to arise when the election officials interact in the 
various stages of processing the votes. We shouldn’t forget about 
those interfaces as well, because that is where we would inadvert- 
ently introduce large inaccuracies. 

Mr. Hoyer. Thank you. 

Let me sort of ask a global question. If we could do only one 
thing this year, do you have any thought as to what that ought to 
be? In other words, the first step. I have mentioned some first steps 
personally, but do you have any views? 

Mr. Woods. Well, from my point of view, it would be very easy 
to put out a kind of initial guide to the human factors of voting 
technology that would give some examples, some basic principles 
and some testing focus group techniques, how to run the focus 
groups, to provide an initial resource to local election officials as 
they contemplate what decisions they should make now or put off 
as more information and options become available. And I think 
there are independent bodies, national laboratories such as 
Brookhaven National Lab or universities, who could respond quick- 
ly to provide that guidance. 

Mr. Hoyer. Any other Comment? 

Mr. Baum. I think if you were only going to do one thing — and 
that is my sincere hope that that is not the case — it would be to 
publish some information on how to live with what you have. I 
mean, yes, there are challenges in each one of these technologies. 
But with proper planning, with good processes in place and good 
training in place, we can certainly reduce the error rate of even the 
equipment that is there by handling it better. 

Mr. Rivest. I think I would respond by saying to create a organi- 
zation whose mission is to assist the States, the local officials, by 
creating a fund, and organization to generate and distribute and 
share information about voting systems, supporting research, sup- 
porting focus groups on usability, supporting hacker attacks on sys- 
tems to see what can be done. You know, publishing a source code, 
when that could be made available. An organization whose goal is 
to help the State officials know everything that needs to be known 
about these systems so they can choose well. 

Mr. Palfrey. I would second that. Because I think one of the 
biggest problems is lack of information, okay? You have election ad- 
ministrators who are in a situation, whether the money is coming 
from the Federal Government or not, they are basically under pres- 
sure. They are going to have to replace some equipment. They don’t 
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have good guidance for how to do it, how to implement it, what 
kind of pitfalls there are. And it goes to security, it goes to all these 
other different issues. I think you really do need an agency or an 
organization who is responsible for collecting and providing an in- 
formation clearinghouse of that sort. 

Mr. Hoyer. As I say and as you know, we do have an organiza- 
tion, small and incorporated within the FEC, the OEA. But, I was 
stricken by the fact that all of the Secretaries of State that testified 
and all of the election administrators that testified were looking to 
the OEA for better information, better — best practices advice and 
counsel, better standards advice. And you have echoed that, and I 
hope we do that. 

I want to say very quickly, I hope we don’t do just one thing. I 
agree with that, and the question was, to that degree, rhetorical. 

Last question I will ask, Mr. Chairman, and I have some others 
here but, if we might, I hope that our staffs can feel free to contact 
you from time to time and get the value of your expertise. 

What would the best way for Congress to facilitate research and 
development of new voting technology, and how can Congress make 
it easier to develop and implement new technology? I think that is 
going to be, I hope, one of the aspects of what we will do. Because 
such a limited market, such a relatively — “sterile” is the wrong 
word but stable market, not a very volatile or vibrant market here, 
so the technology has not turned over as quickly as it otherwise 
might have. Florida has spurred that, and we will have a short 
window, in my opinion, to take advantage of that sort of aware- 
ness. So how can we best do that — that is, to spur our DT&E in 
this area? 

Mr. Baum. I would like to point out that, although the market 
for public elections may be set and stable, there are other kinds of 
elections where this technology applies. There are union elections, 
professional organization elections, and all kinds of places where 
these other factors are also a part. That allows the opportunity for 
the private sector and the public sector to cooperate here in build- 
ing up standards that become then, generally accepted election 
standards. So that is one area of cooperation with the private sec- 
tor where Congress can make a huge contribution. Also, as we have 
all stated before, coming up with standards that can be applied in 
these areas would make a significant contribution. 

Mr. Palfrey. I think one of the barriers to innovation, is that 
election administrators have to be cautious when they make an ac- 
quisition of a replacement for whatever technology they had. And 
what has happened is that they just switch wholesale to a new 
technology. 

I think one thing that would help innovation — and there are 
start-up companies that are out there trying to work on innovation 
in this area — is to provide some sort of funding for pilot testing, 
pilot experiments, and field testing so that you don’t have to jump 
into it full force. You can try it out in a couple precincts, see how 
it is working, and monitor it. I think that is one thing that might 
help. 

Mr. Rivest. I think perhaps some guidance to either NSF or 
DARPA, or both, to support research in this area could be helpful. 
DARPA may seem strange, but in fact there is national security in- 
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volved with these elections. NSF has traditionally funded research 
in computer security and cryptography, for example. As my col- 
leagues have pointed out, there are also businesses that need to be 
involved. I am not sure what the best way to encourage them to 
innovate is. 

Mr. Woods. Another aspect of the activity of the independent re- 
source in that area, we need a mechanism for people to monitor po- 
tential or emerging sources of inaccuracy and problems. This is al- 
most a classic advice that we give to almost every agency that has 
safety, for the potential for a crisis to arise. We don’t want to react 
after the fact to a crisis, like Florida, and try to repair and inter- 
vene. Fligh reliability organizations are out there testing and moni- 
toring their systems to notice early warning signs or even dress re- 
hearsals. Instead of rationalizing away the dress rehearsal of inac- 
curacies in this, our system, which we have been too tolerant of, 
we need these bodies to be out there saying, whoops, look at what 
we are starting to see as problems, sharing that information so 
people can change the technology, change the education, and 
change the procedures they are using to prevent crises from hap- 
pening. 

Mr. Hoyer. Thank you very much to the panelists. You have 
been very helpful, I think, and very thoughtful. Flopefully, as I said 
earlier, we will be able to work with you so you can help us. Again, 
there’s a much longer term — States and localities and whatever 
agency, whether it is the existing OEA within FEC or some other 
organizational structure like that, work with them to accomplish 
objectives which I think clearly all of us want to accomplish. 

And they are not partisan in nature. Everybody wants to make 
sure that, not only does every American have the right to vote, but 
every American’s vote is made easier to cast, more accurately, and 
counted correctly. And although this is not solely a technology 
problem, clearly it manifested itself in many ways as a technology 
problem, as well as a human behavior interface with technology 
problem. 

Thank you very much; and thank you, Mr. Chairman. 

The Chairman. We do appreciate your testimony today. You are 
an important component from the technology side. 

Also one thing, I want to assure you too, and I feel very good 
about, the way we have proceeded and also about the ideas that 
Mr. Hoyer has been able to develop and we have been able to de- 
velop together and, you know, to reach some basic consensus 
among members when you start to talk about it. 

I think the statement you made, Professor Woods, is what we are 
doing — you know, this isn’t debating Florida, but Florida caused 
the debate. And I can barely remember anyone coming to me over 
a period of 20-some years, whatever office I have held, of saying, 
gee, this certain technology needs to be looked at or it needs work. 
It wasn’t drawn out until you had some national view. And that is 
what the Presidential did. 

So I think what we are doing is trying to work with the desires 
of the locals to do something about the existing systems that are 
out there, but also I think what we are doing, is we are not in the 
middle of the crisis right now, we are personally not letting this sit- 
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uation go. We are driving a piece of legislation. I think it is going 
to be good. But I think we are reacting the right way to look ahead. 

So, I think your statements were well taken. It is the way to pro- 
ceed on this, and I think that is what we are doing. 

Mr. Hoyer. Mr. Chairman, on a nonelectoral reform issue, if I 
might, I want to say to you and to Dr. Woods that — how highly re- 
sentful we in Maryland are that you have taken our president of 
the University of Maryland away. I don’t know whether you have 
had an opportunity to get to know Brit Kirwan very well, but he 
is an extraordinary asset. He is a wonderful human being. Ohio 
State is very fortunate to have him. We miss him a lot in College 
Park. 

I am on the board of regents of the university systems, and I was 
a graduate of the University of Maryland and worked with Brit 
very, very closely for all the time that he was at the university 
which, as you know, was over 2 decades. I trust that Ohio appre- 
ciates him as much as they should. You certainly compensated him 
better than we did; and he appreciates that, I know. But I hope you 
enjoy working with Brit. He is a terrific fellow. 

The Chairman. We like him, and we in Ohio will not mention 
what Baltimore took from us in Cleveland a few years ago. We will 
just leave it at that. 

On that note, I ask unanimous consent that witnesses be allowed 
to submit their statements for the record. Members have 7 legisla- 
tive days to insert extraneous material into the record, and for 
those statements and materials to be entered into the appropriate 
place within the record. Without objection, so ordered. 

I ask unanimous consent the staff be permitted to make tech- 
nical and conforming changes on all matters considered, by the 
committee at today’s hearing. Without objection, so ordered. 

Having completed our business of the day for this hearing on 
election reform, the committee is hereby adjourned. Thank you. 

[Whereupon, at 12:35 p.m., the committee was adjourned.] 



